Skip to main content
SpringerLink
Log in

Markov Decision Processes to Model Livestock Systems

  • Chapter
Book cover Handbook of Operations Research in Agriculture and the Agri-Food Industry

Part of the book series: International Series in Operations Research & Management Science ((ISOR,volume 224))

Abstract

Livestock farming problems are often sequential in nature. For instance at a specific time instance the decision on whether to replace an animal or not is based on known information and expectation about the future. At the next decision epoch updated information is available and the decision choice is re-evaluated. As a result Markov decision processes (MDPs) have been used to model livestock decision problems over the last decades. The objective of this chapter is to review the increasing amount of papers using MDPs to model livestock farming systems and provide an overview over the recent advances within this branch of research. Moreover, theory and algorithms for solving both ordinary and hierarchical MDPs are given and possible software for solving MDPs are considered.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    MathWorks Inc. http://www.matlab.com.

  2. 2.

    Oracle http://www.java.com/.

  3. 3.

    R Development Core Team http://www.R-project.org/.

References

  • Allore H, Schruben L, Erb H, Oltenacu P (1998) Design and validation of a dynamic discrete event stochastic simulation model of mastitis control in dairy herds. J Dairy Sci 81(3):703–717. DOI 10.3168/jds.S0022-0302(98) 75626-7

    Article  Google Scholar 

  • van Arendonk J (1985a) A model to estimate the performance, revenues and costs of dairy cows under different production and price situations. Agr Syst 16(3):157–189. DOI 10.1016/0308-521X(85)90010-1

    Article  Google Scholar 

  • van Arendonk J (1985b) Studies on the replacement policies in dairy cattle. ii. optimum policy and influence of changes in production and prices. Livest Prod Sci 13(2):101–121. DOI 10.1016/0301-6226(85)90014-4

    Article  Google Scholar 

  • van Arendonk J (1986) Studies on the replacement policies in dairy cattle. iv. influence of seasonal variation in performance and prices. Livest Prod Sci 14(1):15–28. DOI 10.1016/0301-6226(86)90093-X

    Article  Google Scholar 

  • van Arendonk J (1988) Management guides for insemination and replacement decisions. J Dairy Sci 71(4):1050–1057. DOI 10.3168/jds.S0022-0302(88) 79651-4

    Article  Google Scholar 

  • van Arendonk J, Dijkhuizen A (1985) Studies on the replacement policies in dairy cattle. iii. influence of variation in reproduction and production. Livest Prod Sci 13(4):333–349. DOI 10.1016/0301-6226(85)90025-9

    Article  Google Scholar 

  • van Asseldonk M, Huirne R, Dijkhuizen AA, Beulens A (1999) Dynamic programming to determine optimum investments in information technology on dairy farms. Agr Syst 62(1):17–28. DOI 10.1016/S0308-521X(99) 00051-7

    Article  Google Scholar 

  • Bar D, Tauer L, Bennett G, Gonzalez R, Hertl J, Schukken Y, Schulte H, Welcome F, Groehn Y (2008a) The cost of generic clinical mastitis in dairy cows as estimated by using dynamic programming. J Dairy Sci 91(6):2205–2214. DOI 10.3168/jds.2007-0573

    Article  Google Scholar 

  • Bar D, Tauer L, Bennett G, Gonzalez R, Hertl J, Schulte H, Schukken Y, Welcome F, Grohn Y (2008b) Use of a dynamic programming model to estimate the value of clinical mastitis treatment and prevention options utilized by dairy producers. Agr Syst 99(1):6–12. DOI 10.1016/j.agsy.2008. 09.001

    Article  Google Scholar 

  • Bellman R (1957) Dynamic programming. Princeton University Press, Princeton

    Google Scholar 

  • Ben-Ari Y, Gal S (1986) Optimal replacement policy for multicomponent systems—an application to a dairy-herd. Eur J Oper Res 23(2):213–221. DOI 10.1016/0377-2217(86)90240-7

    Article  Google Scholar 

  • Ben-Ari Y, Amir I, Sharar S (1983) Operational replacement decision model for dairy herds. J Dairy Sci 66:1747–1759. DOI 10.3168/jds. S0022-0302(83)82002-5

    Article  Google Scholar 

  • Boichard D (1990) Estimation of the economic value of conception rate in dairy cattle. Livest Prod Sci 24(3):187–204. DOI 10.1016/0301-6226(90) 90001-M

    Article  Google Scholar 

  • Bono C, Cornou C, Kristensen A (2012) Dynamic production monitoring in pig herds I: modeling and monitoring litter size at herd and sow level. Livest Sci 149(3):289–300. DOI 10.1016/j.livsci.2012.07.023

    Article  Google Scholar 

  • Cabrera V (2010) A large markovian linear program to optimize replacement policies and dairy herd net income for diets and nitrogen excretion. J Dairy Sci 93(1):394–406. DOI 10.3168/jds.2009-2352

    Article  Google Scholar 

  • Cabrera V (2012) A simple formulation and solution to the replacement problem: a practical tool to assess the economic cow value, the value of a new pregnancy, and the cost of a pregnancy loss. J Dairy Sci 95(8):4683–4698. DOI 10.3168/jds.2011-5214

    Article  Google Scholar 

  • Cardoso V, Nogueira J, van Arendonk J (1999a) Optimum replacement and insemination policies for crossbred cattle (holstein friesian x zebu) in the south-east region of brazil. Livest Prod Sci 58(2):95–105. DOI 10.1016/ S0301-6226(98)00205-X

    Article  Google Scholar 

  • Cardoso V, Nogueira J, Van Arendonk J (1999b) Optimal replacement and insemination policies for holstein cattle in the southeastern region of brazil: the effect of selling animals for production. J Dairy Sci 82(7):1449–1458. DOI 10.3168/jds.S0022-0302(99)75372-5

    Article  Google Scholar 

  • Cha E, Hertl J, Bar D, Groehn Y (2010) The cost of different types of lameness in dairy cows calculated by dynamic programming. Prev Vet Med 97(1):1–8. DOI 10.1016/j.prevetmed.2010.07.011

    Article  Google Scholar 

  • Cha E, Bar D, Hertl J, Tauer L, Bennett G, Gonzalez R, Schukken Y, Welcome F, Groehn Y (2011) The cost and management of different types of clinical mastitis in dairy cows estimated by dynamic programming. J Dairy Sci 94(9):4476–4487. DOI 10.3168/jds.2010-4123

    Article  Google Scholar 

  • Dekkers J (1991) Estimation of economic values for dairy cattle breeding goals: bias due to sub-optimal management policies. Livest Prod Sci 29(2–3):131–149. DOI 10.1016/0301-6226(91)90062-U

    Article  Google Scholar 

  • Dekkers JCM, Ten Hag JH, Weersink A (1998) Economic aspects of persistency of lactation in dairy cattle. Livest Prod Sci 53(3):237–252. DOI 10.1016/S0301-6226(97)00124-3

    Article  Google Scholar 

  • Delorenzo M, Spreen T, Bryan G, Beede D, van Arendonk J (1992) Optimizing model: insemination, replacement, seasonal production, and cash flow. J Dairy Sci 75(3):885–896. DOI 10.3168/jds.S0022-0302(92) 77829-1

    Article  Google Scholar 

  • Demeter R, Kristensen A, Dijkstra J, Lansink AO, Meuwissen M, van Arendonk J (2011) A multi-level hierarchic markov process with bayesian updating for herd optimization and simulation in dairy cattle. J Dairy Sci 94(12):5938–5962. DOI 10.3168/jds.2011-4258

    Article  Google Scholar 

  • Fisher W, Schruben L (1953) Linear programming applied to feed-mixing under different price conditions. J Farm Econ 35(4):471–483. URL http://www.jstor.org/stable/1233362

  • Ge L, Kristensen A, Mourits M, Huirne R (2010a) A new decision support framework for managing foot-and-mouth disease epidemics. Ann Oper Res. DOI 10.1007/s10479-010-0774-2

    Google Scholar 

  • Ge L, Mourits M, Kristensen A, Huirne R (2010b) A modelling approach to support dynamic decision-making in the control of fmd epidemics. Prev Vet Med 95(3–4):167–174. DOI 10.1016/j.prevetmed.2010.04.003

    Article  Google Scholar 

  • Giaever H (1966) Optimal dairy cow replacement policies. Ph.D. thesis, University of California, Berkeley, University Microfilms, Ann Arbor

    Google Scholar 

  • Giordano J, Kalantari A, Fricke P, Wiltbank M, Cabrera V (2012) A daily herd markov-chain model to study the reproductive and economic impact of reproductive programs combining timed artificial insemination and estrus detection. J Dairy Sci 95(9):5442–5460. DOI 10.3168/jds.2011-4972

    Article  Google Scholar 

  • Glenn J (1983) A dynamic programming model for pig production. J Oper Res Soc 34:511–519. DOI 10.1057/jors.1983.118

    Article  Google Scholar 

  • Grohn Y, Rajala-Schultz P, Allore H, DeLorenzo M, Hertl J, Galligan D (2003) Optimizing replacement of dairy cows: modeling the effects of diseases. Prev Vet Med 61(1):27–43. DOI 10.1016/S0167-5877(03)00158-2

    Article  Google Scholar 

  • Haran P (1997) Markov decision processes in the optimisation of culling decisions for irish dairy herds. Master’s thesis, School of Computer Applications, Dublin City University

    Google Scholar 

  • Harris B (1990) Recursive stochastic programming applied to dairy cow replacement. Agr Syst 34(1):53–64. DOI 10.1016/0308-521X(90)90093-6

    Article  Google Scholar 

  • Heikkila A, Nousiainen J, Jauhiainen L (2008) Optimal replacement policy and economic value of dairy cows with diverse health status and production capacity. J Dairy Sci 91(6):2342–2352. DOI 10.3168/jds.2007-0736

    Article  Google Scholar 

  • Heikkila A, Nousiainen J, Pyorala S (2012) Costs of clinical mastitis with special reference to premature culling. J Dairy Sci 95(1):139–150. DOI 10. 3168/jds.2011-4321

    Google Scholar 

  • Houben E, Huirne R, Dijkhuizen A, Kristensen A (1994) Optimal replacement of mastitis cows determined by a hierarchic Markov process. J Dairy Sci 77:2975–2993. DOI 10.3168/jds.S0022-0302(94)77239-8

    Article  Google Scholar 

  • Huirne R, Hardaker J (1998) A multi-attribute utility model to optimise sow replacement decisions. Eur Rev Agric Econ 25(4):488–505. DOI 10.1093/ erae/25.4.488

    Article  Google Scholar 

  • Huirne R, Hendriks T, Dijkhuizen A, Giesen G (1988) The economic optimisation of sow replacement decisions by stochastic dynamic programming. J Agric Econ 39:426–438. DOI 10.1111/j.1477-9552.1988. tb00602.x

    Article  Google Scholar 

  • Huirne R, Dijkhuizen A, Renkema JA (1991) Economic optimization of sow replacement decisions on the personal computer by method of stochastic dynamic programming. Livest Prod Sci 28:331–347. DOI 10. 1016/0301-6226(91)90014-H

    Google Scholar 

  • Huirne R, van Beek P, Hendriks T, Dijkhuizen A (1993) An application of stochastic dynamic programming to support sow replacement decisions. Eur J Oper Res 67:161–171. DOI 10.1016/0377-2217(93)90059-V

    Article  Google Scholar 

  • Jalvingh A, Dijkhuizen A, van Arendonk J, Brascamp E (1992a) Dynamic probabilistic modelling of reproductive and replacement in sow herds. general aspects and model description. Agr Syst 39:133–152. DOI 10.1016/ 0308-521X(92)90105-W

    Article  Google Scholar 

  • Jalvingh A, Dijkhuizen A, van Arendonk J, Brascamp E (1992b) An economic comparison of management strategies on reproduction and replacement in sow herds using a dynamic probabilistic model. Livest Prod Sci. 32:331–350. DOI 10.1016/0301-6226(92)90004-N

    Article  Google Scholar 

  • Jalvingh A, van Arendonk J, Dijkhuizen A (1993a) Dynamic probabilistic simulation of dairy-herd management-practices.i. model description and outcome of different seasonal calving patterns. Livest Prod Sci 37(1–2):107–131. DOI 10.1016/0301-6226(93)90067-R

    Article  Google Scholar 

  • Jalvingh A, van Arendonk J, Dijkhuizen A, Renkema J (1993b) Dynamic probabilistic simulation of dairy-herd management-practices. ii. comparison of strategies in order to change a herds calving pattern. Livest Prod Sci 37(1–2):133–152. DOI 10.1016/0301-6226(93)90068-S

    Article  Google Scholar 

  • Jalvingh A, Dijkhuizen A, van Arendonk J (1994) Optimizing the herd calving pattern with linear-programming and dynamic probabilistic simulation. J Dairy Sci 77(6):1719–1730. DOI 10.3168/jds.S0022-0302(94) 77113-7

    Article  Google Scholar 

  • Jørgensen E (1992) Sow replacement: reduction of state space in dynamic programming model and evaluation of benefit from using the model. Dina Research Report 6, National Institute of Animal Science

    Google Scholar 

  • Jørgensen E (1993) The influence of weighing precision on delivery decisions in slaughter pig production. Acta Agric Scand 43(3):181–189. DOI 10. 1080/09064709309410163

    Google Scholar 

  • Kalantari A, Cabrera V (2012) The effect of reproductive performance on the dairy cattle herd value assessed by integrating a daily dynamic programming model with a daily markov chain model. J Dairy Sci 95(10):6160–6170. DOI 10.3168/jds.2012-5587

    Article  Google Scholar 

  • Kalantari A, Mehrabani-Yeganeh H, Moradi M, Sanders A, de Vries A (2010) Determining the optimum replacement policy for holstein dairy herds in iran. J Dairy Sci 93(5):2262–2270. DOI 10.3168/jds.2009-2765

    Article  Google Scholar 

  • Kennedy J (1986) Dynamic programming. Applications to agriculture and natural resources. Elsevier Applied Science Publishers, London/New York

    Google Scholar 

  • Kennedy J, Stott A (1993) An adaptive decision-making aid for dairy cow replacement. Agr Syst 42(1–2):25–39. DOI 10.1016/0308-521X(93) 90066-B

    Article  Google Scholar 

  • Killen L, Kearney B (1978) Optimal dairy cow replacement policy. Ir J Agric Econ Rural Sociol 7:33–40. URL http://www.jstor.org/stable/25556435

  • Kristensen A (1987) Optimal replacement and ranking of dairy cows determined by a hierarchical markov process. Livest Prod Sci 16(2):131–144. DOI 10.1016/0301-6226(87)90015-7

    Article  Google Scholar 

  • Kristensen A (1988) Hierarchic Markov processes and their applications in replacement models. Eur J Oper Res 35(2):207–215. DOI 10.1016/ 0377-2217(88)90031-8

    Article  Google Scholar 

  • Kristensen A (1989) Optimal replacement and ranking of dairy-cows under milk quotas. Acta Agric Scand 39(3):311–318. DOI 10.1080/ 00015128909438523

    Article  Google Scholar 

  • Kristensen A (1992) Optimal replacement in the dairy-herd: a multicomponent system. Agr Syst 39(1):1–24. DOI 10.1016/0308-521X(92)90002-6

    Article  Google Scholar 

  • Kristensen A (1993) Bayesian updating in hierarchical Markov-processes applied to the animal replacement-problem. Eur Rev Agric Econ 20(2):223–239

    Article  Google Scholar 

  • Kristensen A (1994) A survey of Markov decision programming techniques applied to the animal replacement-problem. Eur Rev Agric Econ 21(1):73–93. DOI 10.1093/erae/21.1.73

    Article  Google Scholar 

  • Kristensen A (2003) A general software system for Markov decision processes in herd management applications. Comput Electron Agric 38(3):199–215. DOI 10.1016/S0168-1699(02)00183-7

    Article  Google Scholar 

  • Kristensen A, Jørgensen E (2000) Multi-level hierarchic Markov processes as a framework for herd management support. Ann Oper Res 94:69–89. DOI 10.1023/A:1018921201113

    Article  Google Scholar 

  • Kristensen A, Søllested T (2004a) A sow replacement model using Bayesian updating in a three-level hierarchic Markov process I. Biological model. Livest Prod Sci 87(1):13–24. DOI 10.1016/j.livprodsci.2003.07.004

    Article  Google Scholar 

  • Kristensen A, Søllested T (2004b) A sow replacement model using Bayesian updating in a three-level hierarchic Markov process, II. optimization model. Livest Prod Sci 87(1):25–36. DOI 10.1016/j.livprodsci.2003.07.005

    Article  Google Scholar 

  • Kristensen A, Thysen I (1991a) Economic value of culling information in the presence and absence of a milk quota. Acta Agric Scand 41(2):129–135. DOI 10.1080/00015129109438594

    Article  Google Scholar 

  • Kristensen A, Thysen I (1991b) Ranking of dairy-cows for replacement—alternative methods tested by stochastic simulation. Acta Agric Scand 41(3):295–303. DOI 10.1080/00015129109439912

    Article  Google Scholar 

  • Kristensen A, Nielsen L, Nielsen M (2012) Optimal slaughter pig marketing with emphasis on information from on-line live weight assessment. Livest Sci 145(1–3):95–108. DOI 10.1016/j.livsci.2012.01.003

    Article  Google Scholar 

  • Kure H (1997a) Marketing management support in slaughter pig production. Ph.D. thesis, The Royal Veterinary and Agricultural University. URL http://www.prodstyr.ihh.kvl.dk/pub/phd/kure_thesis.pdf

  • Kure H (1997b) Optimal slaughter pig marketing. In: Proceedings of the dutch/danish symposium on animal health and management economics, Copenhagen, January 23–24 1997. Dina Notat No. 56, 39-47. URL http://www.prodstyr.ihh.kvl.dk/pub/symp/hk/Kure97a.pdf

  • Kure H (1997c) Slaughter pig marketing management: utilization of highly biased herd-specific data. In: Kure et al. (eds.) Proceedings of the first european conference for information technology in agriculture, Copenhagen, June 15–18

    Google Scholar 

  • Langford F, Stott A (2012) Culled early or culled late: economic decisions and risks to welfare in dairy cows. Anim Welf 21(1):41–55. DOI 10.7120/ 096272812X13345905673647

    Article  Google Scholar 

  • Lien G, Kristensen A, Hegrenes A, Hardaker J (2003) Optimal length of leys in an area with winter damage problems. Grass Forage Sci 58(2):168–177. DOI 10.1046/j.1365-2494.2003.00367.x

    Article  Google Scholar 

  • McArthur A (1973) Application of dynamic programming to the culling decision in dairy cattle. Proc N Z Soc Anim Prod 33:141–147

    Google Scholar 

  • Mccullough D, Delorenzo M (1996a) Effects of price and management level on optimal replacement and insemination decisions. J Dairy Sci 79(2):242–253. DOI 10.3168/jds.S0022-0302(96)76357-9

    Article  Google Scholar 

  • Mccullough D, Delorenzo M (1996b) Evaluation of a stochastic dynamic replacement and insemination model for dairy cattle. J Dairy Sci 79(1):50–61. DOI 10.3168/jds.S0022-0302(96)76333-6

    Article  Google Scholar 

  • Mourits M, Huirne R, Dijkhuizen A, Galligan D (1999a) Optimal heifer management decisions and the influence of price and production variables. Livest Prod Sci 60(1):45–58. DOI 10.1016/S0301-6226(99)00037-8

    Article  Google Scholar 

  • Mourits M, Huirne R, Dijkhuizen A, Kristensen A, Galligan D (1999b) Economic optimization of dairy heifer management decisions. Agr Syst 61(1):17–31. DOI 10.1016/S0308-521X(99)00029-3

    Article  Google Scholar 

  • Nielsen B, Kristensen A (2007) Optimal decisions in organic beef production from steers effects of criterion of optimality and price changes. Livest Sci 110(1–2):25–32. DOI 10.1016/j.livsci.2006.09.024

    Article  Google Scholar 

  • Nielsen B, Kristensen A, Thamsborg S (2004) Optimal decisions in organic steer production—a model including winter feed level, grazing strategy and slaughtering policy. Livest Prod Sci 88(3):239–250. DOI 10.1016/j. livprodsci.2003.11.010

    Article  Google Scholar 

  • Nielsen L (2011) Markov decision processes (MDPs) in R. URL https://r-forge.r-project.org/projects/mdp/

  • Nielsen L, Jørgensen E, Kristensen A, Østergaard S (2010) Optimal replacement policies for dairy cows based on daily yield measurements. J Dairy Sci 93(1):77–92. DOI 10.3168/jds.2009-2209

    Google Scholar 

  • Nielsen L, Jørgensen E, Højsgaard S (2011) Embedding a state space model into a markov decision process. Ann Oper Res 190(1):289–309. DOI 10.1007/s10479-010-0688-z

    Article  Google Scholar 

  • Niemi J (2006) A dynamic programming model for optimising feeding and slaughter decisions regarding fattening pigs. Ph.D. thesis, MTT Agrifood Research Finland

    Google Scholar 

  • Noordegraaf A, Buijtels J, Dijkhuizen A, Franken P, Stegeman JA, Verhoeff J (1998) An epidemiological and economic simulation model to evaluate the spread and control of infectious bovine rhinotracheitis in the netherlands. Prev Vet Med 36(3):219–238. DOI 10.1016/S0167-5877(98)00081-6

    Article  Google Scholar 

  • Pihamaa P, Pietola K (2002) Optimal beef cattle management under agricultural policy reforms in finland. Agric Food Sci Finl 11(1):3–12. URL http://jukuri.mtt.fi/handle/10024/452290

  • Pla L, Pomar C, Pomar J (2003) A Markov decision sow model representing the productive lifespan of herd sows. Agr Syst 76(1):253–272. DOI 10. 1016/S0308-521X(02)00102-6

    Google Scholar 

  • Pla L, Pomar C, Pomar J (2004) A sow herd decision support system based on an embedded markov model. Comput Electron Agric 45(1):51–69. DOI 10.1016/j.compag.2004.06.005

    Article  Google Scholar 

  • Powell W (2011) Approximate dynamic programming solving the curses of dimensionality. Wiley, Hoboken

    Book  Google Scholar 

  • Puterman M (1994) Markov decision processes. Wiley series in probability and mathematical statistics. Wiley, New York

    Google Scholar 

  • Rajala-Schultz P, Grohn Y (2001) Comparison of economically optimized culling recommendations and actual culling decisions of finnish ayrshire cows. Prev Vet Med 49(1–2):29–39. DOI 10.1016/S0167-5877(01)00180-5

    Article  Google Scholar 

  • Rajala-Schultz P, Grohn Y, Allore H (2000a) Optimizing breeding decisions for finnish dairy herds. Acta Vet Scand 41(2):199–212

    Google Scholar 

  • Rajala-Schultz P, Grohn Y, Allore H (2000b) Optimizing replacement decisions for finnish dairy herds. Acta Vet Scand 41(2):185–198

    Google Scholar 

  • Rodriguez S, Jensen T, Pla L, Kristensen A (2011) Optimal replacement policies and economic value of clinical observations in sow herds. Livest Sci 138(1–3):207–219. DOI 10.1016/j.livsci.2010.12.026

    Article  Google Scholar 

  • Rogers G, van Arendonk J, McDaniel B (1988a) Influence of involuntary culling on optimum culling rates and annualized net revenue. J Dairy Sci 71(12):3463–3469. DOI 10.3168/jds.S0022-0302(88)79952-X

    Article  Google Scholar 

  • Rogers G, van Arendonk J, McDaniel B (1988b) Influence of production and prices on optimum culling rates and annualized net revenue. J Dairy Sci 71(12):3453–3462. DOI 10.3168/jds.S0022-0302(88)79951-8

    Article  Google Scholar 

  • Smith B (1971) The dairy cow replacement problem. An application of dynamic programming, Bulletin 745. Florida Agricultural Experiment Station, Gainesville

    Google Scholar 

  • Smith B (1973) Dynamic programming of the dairy cow replacement problem. Am J Agric Econ 55(1):100–104. DOI 10.2307/1238671

    Google Scholar 

  • Sørensen J, Kristensen E, Thysen I (1992) A stochastic model simulating the dairy herd on a pc. Agric Syst 39(2):177–200. DOI 10.1016/0308-521X(92) 90107-Y

    Article  Google Scholar 

  • Stewart H, Burnside E, Wilton J, Pfeiffer W (1977) Dynamic-programming approach to culling decisions in commercial dairy herds. J Dairy Sci 60(4):602–617. DOI 10.3168/jds.S0022-0302(77)83908-8

    Article  Google Scholar 

  • Stewart H, Burnside E, Pfeiffer W (1978) Optimal culling strategies for dairy cows of different breeds. J Dairy Sci 61:1605–1615. DOI 10.3168/jds. S0022-0302(78)83772-2

    Article  Google Scholar 

  • Stott A (1994) The economic advantage of longevity in the dairy cow. J Agric Econ 45:113–122. DOI 10.1111/j.1477-9552.1994.tb00382.x

    Article  Google Scholar 

  • Stott A, Kennedy J (1993) The economics of culling dairy-cows with clinical mastitis. Vet Rec 133(20):494–499. DOI 10.1136/vr.133.20.494

    Article  Google Scholar 

  • Stott A, Jones G, Gunn G, Chase-Topping M, Humphry RW, Richardson H, Logue DN (2002) Optimum replacement policies for the control of subclinical mastitis due to s.aureus in dairy cows. J Agric Econ 53(3):627–644. DOI 10.1111/j.1477-9552.2002.tb00041.x

    Article  Google Scholar 

  • Stott A, Jones G, Humphry R, Gunn G (2005) Financial incentive to control paratuberculosis (Johne’s disease) on dairy farms in the United Kingdom. Vet Rec 156(26):825–831. DOI 10.1136/vr.156.26.825

    Article  Google Scholar 

  • Tijms H (2003) A first course in stochastic models. Wiley, West Sussex

    Book  Google Scholar 

  • Toft N, Kristensen A, Jorgensen E (2005) A framework for decision support related to infectious diseases in slaughter pig fattening units. Agr Syst 85(2):120–37. DOI 10.1016/j.agsy.2004.07.017

    Article  Google Scholar 

  • Vargas B, Herrero M, van Arendonk J (2001) Interactions between optimal replacement policies and feeding strategies in dairy herds. Livest Prod Sci 69(1):17–31. DOI 10.1016/S0301-6226(00)00250-5

    Article  Google Scholar 

  • Verstegen J, Sonnemans J, Huirne R, Dijkhuizen A, Cox J (1998) Quantifying the effects of sow-herd management information systems on farmers’ decision making using experimental economics. Am J Agric Econ 80(4):821–829. DOI 10.2307/1244066

    Article  Google Scholar 

  • Viet A, Jeanpierre L, Bouzid M, Mouaddib A (2012) Using markov decision processes to define an adaptive strategy to control the spread of an animal disease. Comput Electron Agric 80:71–79. DOI 10.1016/j.compag.2011.10. 015

    Article  Google Scholar 

  • de Vries A (2004) Economics of delayed replacement when cow performance is seasonal. J Dairy Sci 87(9):2947–2958. DOI 10.3168/jds.S0022-0302(04) 73426-8

    Article  Google Scholar 

  • de Vries A (2006) Economic value of pregnancy in dairy cattle. J Dairy Sci 89(10):3876–3885. DOI 10.3168/jds.S0022-0302(06)72430-4

    Article  Google Scholar 

  • White W (1959) The determination of an optimal replacement policy for a continually operating egg production enterprise. J Farm Econ 41(5):1535–1542. URL http://www.jstor.org/stable/1235311

  • Yalcin C, Stott A (2000) Dynamic programming to investigate financial impacts of mastitis control decisions in milk production systems. J Dairy Res 67(4):515–528. DOI 10.1017/S0022029900004453

    Article  Google Scholar 

  • Yates C, Rehman T (1998) A linear programming formulation of the Markovian decision process approach to modelling the dairy replacement problem. Agr Syst 58(2):185–201. DOI 10.1016/S0308-521X(98)00054-7

    Article  Google Scholar 

Download references

Acknowledgements

The authors are grateful to Dr. Anna-Maija Heikkilä, MTT Economic Research, for her valuable information about several of the Finnish models referenced.

This chapter has been compiled with support from The Danish Council for Strategic Research (The PigIT project, Grant number 11-116191).

Author information

Authors and Affiliations

  1. CORAL, Department of Economics and Business, Aarhus University, Aarhus, Denmark

    Lars Relund Nielsen

  2. Department of Large Animal Sciences, University of Copenhagen, Copenhagen, Denmark

    Anders Ringgaard Kristensen

Authors
  1. Lars Relund Nielsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anders Ringgaard Kristensen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lars Relund Nielsen .

Editor information

Editors and Affiliations

  1. Department of Mathematics Faculty of Law and Economics, University of Lleida, Lleida, Spain

    Lluis M. Plà-Aragonés

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science + Business Media New York

About this chapter

Cite this chapter

Nielsen, L.R., Kristensen, A.R. (2015). Markov Decision Processes to Model Livestock Systems. In: Plà-Aragonés, L. (eds) Handbook of Operations Research in Agriculture and the Agri-Food Industry. International Series in Operations Research & Management Science, vol 224. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-2483-7_19

Download citation

Publish with us

Policies and ethics

Search

Navigation