Recalcitrant dissolved organic nitrogen formation in thermal hydrolysis pretreatment of municipal sludge

Thermal hydrolysis pretreatment (THP) has been considered as an advanced approach to enhance the performance of anaerobic digestion treating municipal sludge. However, several drawbacks were also identified with THP including the formation of brown and ultraviolet-quenching compounds that contain recalcitrant dissolved organic nitrogen (rDON). Melanoidins produced from the Maillard reaction between reducing sugar and amino group have been regarded as a representative of such compounds. This review presented the state-of-the-art understanding of the mechanism of melanoidin formation derived from the research of sludge THP, food processing, and model Maillard reaction systems. Special attentions were paid to factors affecting melanoidin formation and their implications to the control of rDON in the sludge THP process. These factors include reactant availability, heating temperature and time, pH, and the presence of metallic ions. It was concluded that efforts need to be focused on elucidating the extent of the Maillard reaction in sludge THP. This paper aims to provide a mechanistic recommendation on the research and control of the THP-resulted rDON in municipal wastewater


Introduction
Thermal hydrolysis pretreatment (THP) is known as an effective municipal sludge pre-treatment process for enhancing anaerobic digestion (Barber 2016). In recent years, increasing applications of sludge THP have been reported in full-scale water resource recovery facilities (WRRFs). Seventy-five full-scale THP facilities are currently either in operation or in planning across the world treating 1.65 million metric dry tons of sludge per year (Barber 2016). The principle behind THP involves the hydrolysis and solubilization of the organic component of municipal sludge at elevated temperatures and pressures, commonly reported in the ranges of 160-190°C and 480-1260 kPa (4.8-12.6 bar), respectively (Wilson et al., 2011). The advantages of THP on sludge treatment have been well documented including: (i) reduction of sludge viscosity so that the sludge with higher solids content, e.g. 7-9% total solids (TS), can still be pumped and mixed in anaerobic digesters (Bougrier et al., 2006, Oosterhuis et al., 2014; (ii) improvement of sludge digestibility and thus biogas production (Liu et al., 2012;Stuckey and McCarty 1984); (iii) enhancement of sludge dewaterability (Higgins et al., 2017;Phothilangka et al., 2008); and (iv) pathogen sterilization and odor reduction (Murthy et al., 2009;Neyens and Baeyens 2003). Along with the advantages THP brings, there also comes the disadvantages that have not been fully resolved. For example, THP results in the production of substances with high color and ultraviolet (UV)-quenching ability, the inhibition of side-stream nitrogen removal processes such as nitrification and anammox, as well as the generation of recalcitrant carbon, nitrogen, and phosphorus species (Gu et al., 2018). Among these disadvantages, the production of recalcitrant dissolved organic nitrogen (rDON) from THP is selected as a major focus of this review because rDON is known to not only deteriorate the efficiency of UV-disinfection through UV-quenching but also substantially increase the organic fraction of total nitrogen (TN) in the discharge of WRRFs. Dwyer et al. (2008b) has reported a 18% decrease of UV-transmission and a 50% increase of effluent rDON concentration from 1.8 to 2.8 mg L −1 based on the historical data of a full-scale WRRF after commissioning sludge THP. It should be realized that the effluent TN limits of 3 mg L −1 or less are common in regions such as the Chesapeake Bay watershed, the coastal areas of North Carolina, and mid-Colorado in the U.S., and Okanagan Lake area of British Columbia in Canada, and increasingly stringent effluent discharge limits will be imposed in the near future (Moore 2010). For example, the 2000 Chesapeake Bay agreement mandated 48% reduction of TN loads from https://doi.org/10.1016/j.envint.2020.105629 Received 5 December 2019; Received in revised form 7 February 2020; Accepted 1 March 2020 WRRFs based on the 1990 levels, which has led to more stringent effluent TN limits as low as 3 mg L −1 by 2011 (Mulholland et al., 2007). By 2017, only 40% of the nitrogen reduction goal was achieved, which has failed to meet the midpoint-goal of 60% of the necessary TN reduction (EPA, 2019). Clearly, as more WRRFs apply THP, the substantial rDON increases observed after THP implementation, e.g. 50%, can become a real threat for WRRFs to meet the increasingly stringent TN limits (Dwyer et al., 2008b). Therefore, serious considerations need to be taken towards the possible rDON formation as a result of THP.
The study by Stuckey and McCarty (1984) was probably the first report of rDON production from THP of activated sludge. A considerable reduction of nitrogen compounds biodegradability was observed in THP operated at 150 to 200°C, even though the biodegradability of the treated activated sludge was significantly improved. Penaud et al. (2000) further characterized the recalcitrant soluble compounds produced from the THP of microbial biomass, and then related the recalcitrant soluble compounds to the products from Maillard reaction based on the similarities in terms of supernatant color, product molecular weight, biodegradability, as well as the influence of acid precipitation. More recently, increased color, decreased UV-transmission, and increased dissolved organic nitrogen (DON) concentration in plant effluent after the implementation of sludge THP have been reported in a full-scale biological nutrient removal system located at Queensland, AU (Dwyer et al., 2008b). The study conducted within the full-scale WRRFs further inferred that the Maillard reaction products, e.g. melanoidins, may be responsible for the rDON production during the sludge THP based on the comparisons of color, UV-quenching, dissolved organic carbon (DOC), DON, and molecular weight between the THP effluent, the synthetic melanoidin solution, and the plant effluents from the WRRF with and without THP treated sludge centrate return. As a newly identified issue, an in-depth understanding of the rDON formation mechanism during THP is essential to ensure a sustainable application of THP. Therefore, in this review, the mechanism of Maillard reaction in the context of THP is discussed to provide a technical guidance for rDON control and also for future research need identification.

What is the Maillard reaction and why it is relevant to rDON formation in sludge THP?
It has been recognized that the characteristics between the synthetic melanoidins formed from the Maillard reaction and the nitrogen-containing, macromolecular, dissolved substances separated from THP effluent are very similar. These characteristics include molecular weight, DON and DOC contents, color, UV-quenching, fluorescence, aromaticity, etc. (Ahuja et al., 2015;Dwyer et al., 2008b, Gupta et al., 2015Higgins et al., 2017;Penaud et al., 2000). Hence, it is important to understand the principle of the Maillard reaction that dictates the production of melanoidins, a primary type of rDON.
The Maillard reaction is named after French chemist Louis-Camille Maillard who first described the reaction in 1912 (Maillard 1912). The Maillard reaction is a non-enzymatic browning reaction that occurs between reducing sugar and amino group at an elevated temperature, forming dark-colored, UV-quenching, and hardly biodegradable polymers (Hodge 1953;Maillard 1912;Reynolds 1965). Another similar non-enzymatic browning reaction also contributing to the color development is sugar caramelization which usually occurs at a low water activity (Buera et al., 1987). Since the caramelization reaction does not involve nitrogen and thus is not covered in in this review.
Although the Maillard reaction was discovered one hundred years ago, it was not until recently that its implication in the THP of municipal sludge was partially understood. In fact, most researches related to the Maillard reaction were performed in the field of food and flavor industry where the reaction is intentionally facilitated to provide brown color and generate flavors and aromas of cooked foods (Martins et al., 2000). Since the typical THP conditions, e.g. 165°C for 30 min, overlap with that of the Maillard reaction, and municipal sludge is full of polysaccharides (20−40%) and proteins (30−50%) as potential reactants for Maillard reaction, the knowledge derived from the food industry provides a mechanistic basis for understanding and controlling the rDON production during the THP of municipal sludge (Jimenez et al., 2013).
Nitrogen-containing recalcitrant organics can be produced through the Maillard reaction via the chemical reaction between the carbonyl groups of reducing sugar and the amino groups of amino acids, peptides, or proteins (Hodge 1953). The chemistry underlying the Maillard reaction is complex due to the large variety of reaction pathways including condensation, cyclisations, dehydrations, retroaldolisations, rearrangements, and isomerisations, and their strong dependence on reaction conditions such as temperature, time, pH, and composition of reactants (Labuza et al., 1998;Martins et al., 2000). For this reason, Maillard reaction products are believed to be a mixture of heterogeneous compounds including those with relatively low molecular weights (LMWs, < 3.5 kDa) such as aldehydes, ketones, dicarbonyls, acrylamides, heterocyclic amines, and those polymeric compounds with large molecular weights (HMWs, > 10 kDa) such as advanced glycation end-products .
Color and UV-quenching are distinct characteristics of melanoidins. Many attempts have been made to quantify the color development and UV-quenching of melanoidins in model Maillard systems by using spectrophotometric approach and the Lambert-Beer equation in Eq. (1) Kim and Lee, 2008b;Rufián-Henares and Morales, 2007): in which A is the light/UV absorbance, ε is the extinction coefficient at a given wave length, l is the length of light/UV passing through in solution, and c is the concentration of absorbent. For example, Rufián-Henares and Morales (2007) determined the ε in Eq.
(1) at 420 nm for glucose-tryptophan and glucose-lysine melanoidins to be 0.225 and 4.315 ml mg −1 cm −1 , respectively. However, for complicated reactants such as real food and sludge, Eq. (1) and ε become less useful due to the complicated chemical composition and thus the unknown concentrations of each component, e.g. c in Eq.
(1), of melanoidins formed with those reactants. Instead, the increase in light or UV absorbance (A) before and after THP was often directly used as an indicator for melanoidin production (Ahuja et al., 2015;Dwyer et al., 2008b;Higgins et al., 2017;Penaud et al., 2000). With regard to the nitrogen content in Maillard reaction products, Cämmerer and Kroh (1995) reported that the C:N ratio varies from 7.4 to 26.3 when different reducing sugars reacted with glycine following a molar ratio of 1 to 1. Ahuja et al. (2015); Dwyer et al. (2008b), andHiggins et al. (2017) reported substantial DON increase after sludge THP likely due to the production of melanoidins, and the DON production has been observed to increase with the THP operating temperature. As recalcitrant organic compounds, melanoidins are also known for poor microbial degradability and even adverse biological effects including genotoxicity, cytotoxicity, and antimicrobial activity (Chandra et al., 2008;Wang et al., 2011). Ivarson and Benzing-Purdie (1987) evaluated the biodegradation of U-14 C labeled synthetic melanoidins with soil microorganisms and reported only 1 mg out of 1.42 g reduction of melanoidins after aerobic incubation for 25 days. For sludge THP, Stuckey and McCarty (1984) reported that the anaerobic biodegradability of mixed amino acids decreased 18-22% after the THP at 200°C in comparison with the untreated samples, despite the overall biodegradability improvement of the activated sludge. Penaud et al. (2000) reported that the removal of HMW melanoidins by resin decolorization or acid precipitation led to an improvement of sludge anaerobic biodegradability by 26% in terms of biogas conversion. In addition, Gupta et al. (2015) reported that the aerobic biological treatment had little effect on the organic nitrogen in the returned liquor collected from a full-scale plant sequentially processed with THP, anaerobic digestion, and dewatering. These observations are closely related to the characteristics of melanoidins as summarized in Table 1.

What factors affect the Maillard reaction and rDON production?
In this section, we present internal and external factors that affect both the rate and extent of the Millard reaction, with the hope of providing a fundamental understanding for developing strategies to control the Millard reactions and formation of rDON in THP conditions. Four predominating reaction factors, including reactants, heating temperature and time, pH, and the presence of metallic ions are discussed to reveal their impact on the Maillard reaction and the properties of derived rDON products.

Effect of reactants
Municipal sludge is rich of polysaccharides (20% − 40%) and proteins (30% − 50%), providing abundant reactants for the Maillard reaction at THP conditions (Jimenez et al., 2013). For both pure reactants such as reducing sugars and amino acids used in model systems and real biomass such as food products, the properties of melanoidins formed were found to be reactant specific (Bekedam et al., 2008a;Bekedam et al., 2008b;Cämmerer and Kroh, 1995;Ortega-Heras and González-Sanjosé, 2009; Rufián-Henares and Morales, 2007;Van Chuyen et al., 1973a;Van Chuyen et al., 1973b). As mentioned previously, the Maillard reactions between glycine and six different Dcarbohydrates yielded melanoidins with distinct relative content of DON, i.e. C:N ratios varying from 7.4 to 26.3 under the same reaction condition (Cämmerer and Kroh, 1995). However, the results from the same study also indicated that the molar ratio of the reactants had little effect on the C:N ratio of the melanoidins produced (Cämmerer and Kroh, 1995). The browning effect, e.g. the light absorbance of melanoidins purified from different reactants, also showed significant differences. For example, ε in Eq. (1) determined for glucose-tryptophan and glucose-lysine melanoidins has been reported to be of a 19-fold difference with values of 0.225 and 4.315 ml mg −1 cm −1 , respectively (Rufián-Henares and Morales, 2007).
The reactivity of reactants was also reported to affect the kinetics of the Maillard reaction and in turn the characteristics of melanoidins formed. For example, Van Chuyen et al. (1973a, 1973b quantified the rate of amines degradation and color development at 80°C when glyoxal reacted with amines with reducing peptides amount, i.e., in the order of three (tetraglycine), two (triglycine), one (diglycine), and zero (glycine) peptides. It showed that the reactivity of the peptides, namely amino acids with short chain, was much greater than that of the single amino acid in terms of color development and amine utilization rates.
The compositions and characteristics of melanoidins in real biomass are more complex than those in model systems due to its diverse pool of reactants. Many studies in food science have demonstrated that specific food-originated compounds may play important roles in formulating melanoidin compositions. For example, the presence of phenolics in coffee melanoidins has been largely attributed to the presence of chlorogenic acids in coffee beans (Bekedam et al., 2008a, Bekedam et al., 2008b. Polyphenols, the natural micronutrients found in grape, have been shown to contribute to the melanoidins of grape syrup and sweet wine (Ortega-Heras and González-Sanjosé, 2009). For municipal sludge THP, the effect of reactants on melanoidin production is largely unknown. Further investigations, for example on the extent of reducing sugar production and the reactivity of various amino groups in sludge, are important to assess the melanoidin production during THP.

Effect of heating temperature and time
Since the discovery of the Maillard reaction, heating temperature and time have been major factors investigated. The kinetics of the Maillard reaction can be considered as three steps as described in Eqs.
(i) Early-stage: (ii) Intermediate stage: (iii) Advanced stage: Temperature plays an important role in determining the Maillard reaction rate constant of each step and in turn governs the dominating reaction pathways and the degree of polymerization and/or crosslinking. Based on the study by Ge and Lee (1997), using a phenylalanine-glucose model system, the formation of Schiff base complex rather than the formation of Amadori products was determined to be the ratelimiting step during the early stage of the Maillard reaction (Eq. (2)) indicated by three orders of magnitude higher of k 1 than k 2 , namely 3.97 and 3.54 × 10 −3 h −1 at the temperature of 97°C, respectively.
It is known that the effect of temperature on reaction rate constant can be described by using the well-known Arrhenius equation, which shows that reaction steps associated with higher activation energy (E a ) are more sensitive to temperature changes (Martins et al., 2000). Ge and Lee (1997) observed that the formation rate of Amadori products from phenylalanine and glucose in Eq. (2) with an E a of 33.5 kJ mol −1 started to slowly increase at a temperature around 70°C, and then sharply increased beyond 90°C, while the formation rate of Schiff base complex in Eq. (2) with an E a of 27.3 kJ mol −1 increased much slower with temperature. During the intermediate stage in Eq. (3), k 3 (first order) and k 4 (second order) in Eq. (3) for intermediate and LMW melanoidins formation were in the range of 0.6 to 4 × 10 −2 h −1 and 0.7 to 21.6 mol −1 L h −1 , respectively, which are much greater than the kinetic constants of the early stage in Eq. (2), e.g., 0.7 to 3 × 10 −4 mol −1 L h −1 as determined for glucose reacting with lysine, arginine, glycine, serine, valine, glutamic acid, aspartic acid, or alanine (Wedzicha and Leong, 2005). Despite the large variety of reactants and pathways in the advanced stage of the Maillard reaction (Eq. (4)), the formation kinetics of HMW melanoidins was often reported as zero order , Morales and Van Boekel, 1996, Morales et al., 1995, Van Boekel, 2001.  reported that the kinetic constant in the advanced stage (Eq. (4)) was the greatest among all reaction steps with a k 5 of 1.14 h −1 and E a of 128 kJ mol −1 for glucose-lysine and a k 5 of 1.86 h −1 and E a of 75 kJ mol −1 for fructose-lysine systems. It should be noted that most of the Maillard reaction kinetic constants in the aforementioned studies were obtained at relatively low temperatures e.g. 70-100°C, and substantially higher reaction kinetic constants can be expected under the typical THP temperature, e.g. 165°C, according to the Arrhenius equation. At a temperature above 200°C, the decomposition of some melanoidins to bio-char, bio-oil, gas, ammonia, and other decomposed products has been reported in studies related to hydrothermal liquefaction process (Minowa et al., 2004, Wang, 2011, Yang et al., 2015. With those being said, the first step of the Maillard reaction, i.e., the formation of Schiff base complex from amine and sugar (k 1 in Eq. (2)), is usually believed to be the rate-limiting step for melanoidin production. Consequently, further regulation of this rate-limiting step may serve as an efficient measure for rDON control. This can be potentially achieved by reducing the availability of the essential reactants, e.g. amines, through pH adjustment or amine wrapping by coagulants. It has been reported that the Amadori product formation rate in Eq. (2) exponentially decreased with pH decrease, which is in line with the effect of pH on the concentration of reactive un-protonated amino groups (Ge and Lee, 1997).
In addition to kinetic constants, temperature also affects the molecular weight, biodegradability, color, and element compositions of the melanoidins formed. Wang et al. (2011) summarized that a heating temperature lower than 50°C and a duration more than 30 days produced mainly LMW (< 12-14 kDa) melanoidins during the fermentation and storage of beer, sweet wine, and grape syrup. In contrast, a heating temperature higher than 150°C and a duration less than 2 h produced HMW (> 12-14 kDa) melanoidins during the production of bread, coffee, roasted malt, cocoa, and biscuits. Notably, temperature increase was also reported to decrease the biodegradability of synthesized melanoidins by both aerobic and anaerobic microorganisms (Ivarson and Benzing-Purdie, 1987), which is undesired for WRRFs with a high effluent TN standard. The color formation is more associated with the formation of LMW and HMW melanoidins in Eqs. (3) and (4) (Wedzicha and Leong, 2005). The darkness of HMW melanoidins has been reported to increase with increasing heating temperature and time , Echavarría et al., 2013, Fogliano et al., 1999. Moreover, Motai (1974) investigated the molecular weights of the color components in glycine-xylose melanoidins, and concluded that the darkness of melanoidins was increased by the polymerization of LMW melanoidins to form HMW melanoidins in the advanced stage (Eq. (4)). Finally, it was also revealed that temperature may alter the element compositions of melanoidins. For example, Cämmerer and Kroh (1995) reported that a temperature increase from 60 to 170°C increased the C:N ratio from 6.2 to 12.5 for glucose/glycine melanoidins. For sludge THP, operating temperature increases, e.g. from 140 to 165°C, have been demonstrated to lead to more melanoidin production, as indicated by the increased darkness, UV-quenching, DON, and organic recalcitrance (Ahuja et al., 2015, Dwyer et al., 2008b, Higgins et al., 2017. Unfortunately, systematic investigations of the effects of temperature on the characteristics of melanoidins formed during sludge THP remain absent.

Effects of pH
pH has been demonstrated to be an important factor that influences the reactant reactivity and the structure of the melanoidins. For example, pH determined the extent to which the reactions proceed in Eq.
(3), e.g. the formation of reductones (pH > 7), Fission products (pH = 7), or Schiff's base of furfural (pH < 7) (Martins et al., 2000). The fundamentals of the effect of pH on the Maillard reaction have to do with the changes in the availability of the reactive forms of reducing sugar and amino group in response to pH changes. The un-protonated form of amino group and the open-chain form of reducing sugar at higher pH are considered to be the reactive forms (Martins et al., 2000). Depending on the pKa of a specific amino group-containing compound, as pH decreases, amino groups will be protonated and gradually lose their reactivity with reducing sugar as shown in Eq. (5), To better illustrate the availability of the reactive forms of amino group at different pH, Fig. 1a illustrates an example of the effect of pH on the availability of un-protonated glycine which has a pKa of 9.6. When the pH is below 7, the content of reactive amino group is less than D. Zhang, et al. Environment International 138 (2020) 105629 0.3%, and the reactive amino group content increases exponentially with pH (Martins et al., 2000). Fig. 1b shows the effect of pH on the formation rate of the phenylalanine Amadori compound, namely a product of the early-stage Maillard reaction in Eq.
(2). Again, as pH increased towards alkaline condition, the proton releasing from the amine ion increased the reactive amino acid content, leading to an exponential increase of the Amadori formation rate (Ge and Lee, 1997). The reaction pathways and products of Millard reaction also depend on the types of reducing sugar involved and pH. In aqueous solution, reducing sugar presents in several chemical configurations, including open chain form, cyclic form, ionized form, enediol anion (enolization), and isomers in equilibrium (Martins et al., 2000). It has been reported that the degree of sugar ionization, enolization, and isomerisation increased as pH increased, which further result in different color patterns of melanoidins formed after the Millard reaction (Kim and Lee, 2008a, Laroque et al., 2008, Martins et al., 2000. More thorough studies are still needed to further demonstrate the dependence of melanoidin properties on sugar structure and explain the mechanisms. It should be noted that the pH usually drops as a result of THP, probably because of the formation of reductone and fatty acids along with the development of the Maillard reaction. The drop of pH has recently been confirmed in glucose-lysine system without buffer (Han et al., 2017, Kwak et al., 2005, Wang et al., 2009. Consistent with the aforementioned pH effect, less dark and polymerized melanoidins were formed from the same reactants when pH dropped without buffer addition (Kwak et al., 2005). Thus, for sludge THP, not only the initial sludge pH, but also the pH dynamics in the course of THP is important for controlling the rDON production.

Effect of metallic ions
Metallic ions such as magnesium, calcium, aluminum, and iron are commonly used in the municipal sludge handling processes to assist sludge thickening, phosphorus precipitation, and odor mitigation (Dassey and Theegala, 2012;Morse et al., 1998;Park and Novak, 2013;Zhang et al., 2020). As anionic compounds, melanoidins and some Maillard reaction intermediates have been reported to bind with metallic ions. However, the effect of metallic ion binding on the color development of melanoidins is still under debate. O'brien and Morrissey (1997) have reported that the Amadori compounds formed at the earlystage (Eq. (2)) of the Maillard reaction were able to bind metallic ions with varying binding strength in the order of Mg 2+ > Cu 2+ = Ca 2+ > Zn 2+ . Gomyo and Horikoshi (1976) showed that melanoidins have a remarkable coagulation capacity with various metallic ions including Fe 3+ , Al 3+ , Cu 2+ , Zn 2+ , Co 2+ , and Mn 2+ , and their browning was suppressed by metallic ion additions. However, Morales et al. (2005) examined the iron (II)-binding ability of several food-originated melanoidins and found no relationship between browning and iron binding ability of melanoidins.
On the contrary, Morita and Kashimura (1991) proposed that transition metallic ions such as Cu 2+ , Fe 2+ , Fe 3+ , and Mn 2+ were able to catalyze the Maillard reaction and promote the formation of chromophores through oxidative pathways indicated by the 48% − 107% increases in 330 nm absorbance with metal additions. The study by Kwak and Lim (2004) found that the presence of metallic ions can either accelerate or inhibit browning depending on what kinds of amino acids involved, and the transition metals Cu 2+ and Fe 2+ accelerated the browning of melanoidins the most in general.
Apparently, various mechanisms including chelation, catalyzation, and coagulation may be involved in the interaction of melanoidins and metallic ions. Additionally, the metal binding of melanoidins has also been related to the antioxidant and antimicrobial properties of melanoidions (Rufián-Henares and de la Cueva, 2009.

Current understanding of the Maillard reaction in THP of municipal sludge
The organic components of municipal sludge (about 80% as volatile solids) typically contain polysaccharides (20−40%), proteins (30−50%), and lipids (< 10%), which provides a pool of reactants to fuel the Maillard reaction (Jimenez et al., 2013). Moreover, the heating conditions of THP, for example 165°C for 30 min in the commercialized THP processes (e.g. CAMBI TM ), overlap with the conditions ideal for the formation of melanoidins through the Maillard reaction (Barber, 2016). Despite the high possibility for the Maillard reaction to take place in the THP of municipal sludge, the study of the Maillard reaction in the field of wastewater treatment is still in its early stage.
Many studies have related the increase of DON, browning, and enhanced UV-quenching in the effluent of sludge THP or downstream anaerobic digestion to the formation of melanoidins as summarized in Table 1 (Ahuja et al., 2015;Dwyer et al., 2008b;Gupta et al., 2015;Higgins et al., 2017;Penaud et al., 2000;Wilson and Novak, 2009). Wilson and Novak (2009) further reported that the THP of polysaccharides alone in a model system at a temperature below 190°C resulted in little UV-quenching, suggesting sugar browning through caramelization was not evident at the typical temperature of THP for polysaccharides. Despite these evidences, little information has been provided to describe to what extent the organics in sludge have participated in the Maillard reaction and formed melanoidins. This is particularly important because THP aims to increase the biodegradability of sludge by hydrolysis of the macromolecular organics, while the Maillard reaction and the formation of highly recalcitrant melanoidins play an opposite role. Most studies regarding the Maillard reaction reviewed in this paper were at temperatures below 200°C. As mentioned Fig. 1. (a) Effect of pH on the reactive form of the amino group of glycine (Martins et al., 2000); (b) Effect of pH on the formation rate of phenylalanine Amadori compound (Ge and Lee 1997). Zhang, et al. Environment International 138 (2020) 105629 previously, recalcitrant melanoidins tend to form with a much faster kinetics at a temperature above 90°C, and the reaction rate increases with temperature. The study by Higgins et al. (2017) showed an increase of DON concentration in the anaerobic digester effluent from 380 to 470 mg/L as a result of the increase of THP temperature from 130 to 160°C, suggesting the well-accepted knowledge of enhanced melanoidins formation at high temperatures Echavarría et al., 2013;Fogliano et al., 1999;Ge and Lee, 1997). However, studies showed that sludge biodegradability, as indicated by the change of methane yield before and after THP, also increased with THP temperature starting from 100°C until a maximum level was reached in the temperature range of 175 to 190°C (Higgins et al., 2017). From a practical perspective, this indicated the Maillard reaction will occur in parallel with thermal hydrolysis of solids, and the impacts of melanoidins formation must be balanced with the benefits associated with the improvement to solids reduction and biogas production. At a temperature above 190°C, the sludge biodegradability has been reported to decrease, and this has been also related to the Maillard reaction in previous studies (Bougrier et al., 2008;Pinnekamp 1988;Stuckey and McCarty 1984). However, as revealed in the work of Wilson and Novak (2009) using polysaccharides, the effect of sugar caramerlization should be taken into consideration and compared with the effect of the Maillard reaction at the temperature above 190°C to better explain the contribution of the Maillard reaction to the observed sludge biodegradability decrease. Furthermore, very few studies directly investigated the fate of THP-generated melanoidins and rDON in anoxic and aerobic conditions which are typically employed during mainstream wastewater treatment. Previous studies on the effect of THP on rDON production by Higgins et al. (2017) and Dwyer et al. (2008b) only examined the DON concentration in THP or anaerobic digester effluent but did not test the aerobic biodegradability of DON. It should be realized that DON from THP or anaerobic digester effluent may not be adequate to represent rDON. Further work is also needed to investigate the recalcitrance of DON in aerobic and anoxic conditions similar to the mainstream treatment. Furthermore, only a little work was performed to characterize melanoidins formed from sludge THP, and no study has been conducted to address the effect of sludge type and composition, e.g. primary, secondary, or chemical sludge. Dwyer et al. (2008b) compared the molecular weight, DON and DOC contents, color, UV-quenching, fluorescence, and aromaticity between the synthetic melanoidins (glucose-glycine) and the macromolecular dissolved substances separated from THP effluent and also the effluents of two full-scale WRRFs with and without THP. Similarities were identified for the compounds with molecular weight above 10 kDa. However, biodegradability test was not carried out to verify the recalcitrance of THP effluent or to confirm that melanoidins can fully represent the increased rDON in plant effluent. Penaud et al. (2000) reported that the browning effect was mostly provided by the compounds with molecular weights of 9 to 28 kDa, while the compounds with molecular weights > 100 kDa contributed the most to the low biodegradability of THP sludge. Studies from other fields demonstrated that the property of melanoidins such as color, molecular weight, and nitrogen content can be highly reactant specific as discussed in Section 3.1 (Cämmerer and Kroh 1995;Rufián-Henares and Morales 2007). In order to better evaluate the significance of the Maillard reaction in sludge THP and have a better understanding of sludge-originated melanoidins, the effect of various chemical components of primary treatment, secondary treatment, etc., on melanoidin production in THP should be studied. For example, comparing primary to secondary sludge in the U.S., the former usually contains more volatile solids (75% vs 70% of TS) and polysaccharide (44% vs 26% of TS), less protein (25% vs 36% of TS) and nitrogen (2.5% vs 3.8% TS), and more acidic pH (6 vs 7.1) (Burton et al., 2013). This implies that the THP of secondary sludge may be liable to a higher rDON production potential than the THP of primary sludge due to the higher protein contents and pH. In addition, melanoidins have been demonstrated to have multiple biological effects such as genotoxicity, cytotoxicity, antioxidative effect, and antimicrobial activity (Chandra et al., 2008;Wang et al., 2011). These toxic effects may contribute to the observed inhibition of side-stream nitrogen removal process receiving THP return flow (Gu et al., 2018). Currently, the topic of the biological effect of melanoidins are absent in the study of sludge THP, and whether these properties impact the microbial activity in the downstream anaerobic digestion and biological nitrogen removal is also unknown.
Finally, the effects of THP operational conditions other than temperature and time, such as moisture, pressure, pH, buffer capacity, and metal addition on rDON generation have not been investigated. pH is an important factor determining the reactivity of reducing sugar and amino group. It has been reported that THP can reduce sludge pH and increase the sludge alkalinity, likely due to the formation of reductone and the release of weak acids such as fatty acids (Han et al., et al., 2017, Wang et al., 2009. Thus, the Maillard reaction is not only affected by the initial pH of the sludge but also by the dynamics of pH during THP (Han et al., 2017;Wang et al., 2009). Besides, some sludge THP has been hybridized with acid and alkaline treatment, which potentially has a more substantial effect on melanoidin formation due to the more extreme pH conditions (Neyens et al., 2003a;Neyens et al., 2003b;Shehu et al., 2012;Vlyssides and Karlis 2004). Although the effect of metallic ions on the Maillard reaction is still unclear, significant changes of melanoidin production with and without metallic ion additions have been observed (Gomyo and Horikoshi 1976;Kwak and Lim 2004;Morita and Kashimura 1991;O'brien and Morrissey (1997)). Both transition (Fe) and non-transition (Al, Ca, Mg) metallic ions are commonly used in WRRFs (Dassey and Theegala, 2012;Morse et al., 1998;Park and Novak, 2013;Zhang et al., 2020). Thus, their impacts on rDON formation during sludge THP are worth investigation.

Potential strategies for rDON control during and after sludge THP
Melanoidins may appear following THP and create operational problem on UV-disinfection and biological nutrient removal due to their recalcitrant and UV-quenching nature. Thus, strategies for melanoidin reduction are desired in the application of THP in WRRFs.

Lower THP temperature
Theoretically, a lower temperature will result in lower reaction coefficients (Eqs. (2)-(4)), especially for the rate limiting step (k 1 ) in Eq. (1), hence provides a possible strategy to reduce melanoidin and rDON production. Many researches have reported that a low THP temperature (e.g. 130-150°C) can reduce UV-quenching in the anaerobic digestion effluents, however doing so was also reported to reduce the sludge digestibility (Higgins et al., 2017;Wilson and Novak, 2009). Differently, the study by Dwyer et al. (2008b) suggested that low THP temperature (140-165°C) reduced the production of melanoidins but left the sludge digestibility unaffected. These reports imply a potential trade-off between sludge digestibility improvement and the reduction of melanoidins by decreasing the THP temperature. Homma et al. (1982) measured the isoelectric points of various Maillard reaction products, at which these compounds can be precipitated. The results showed that all Maillard reaction products evaluated showed isoelectric points around pH = 3. Later, Penaud et al. (2000) studied the performance of acid precipitation at the pH of 3 in precipitating melanoidins from sludge THP effluent and observed considerable improvement (over 20%) in sludge digestibility after the acid treatment. Although adjusting pH to this extreme level (pH = 3) is not feasible in practice to control melanoidins, the reactivity of amino group remarkably declines at pH below 7 (Fig. 1), which indicated that D. Zhang, et al. Environment International 138 (2020) 105629 pH adjustment and control within a narrow range around 5-7 may also lead to effective melanoidin reduction. Further investigation is warranted to verify this hypothesis.

Upstream metallic ion addition
Metallic ions such as magnesium, calcium, aluminum, and iron are commonly used in wastewater treatment to assist sludge thickening, odor prevention, and/or phosphorus precipitation (Dassey and Theegala, 2012;Morse et al., 1998;Park and Novak, 2013;Zhang et al., 2020;Burton et al., 2013). As mentioned previously, some of the metallic ions, e.g. Al 3+ and Ca 2+ , bind melanoidins, leading to rDON reduction after THP, yet some of the transition metallic ions, e.g. Fe 3+ and Mn 2+ , catalyze the Maillard reaction and lead to increased rDON production (Gomyo and Horikoshi 1976;Morita and Kashimura 1991). Therefore, selection of right metallic coagulants in the processes prior to sludge THP may also serve as a potential strategy for melanoidin control. For example, substitution of Fe 3+ with Al 3+ coagulant prior to sludge THP offers possibility to substantially reduce melanoidin production.

Sludge conditioning and dewatering
The anaerobically digested sludge following THP is often subjected to conditioning and dewatering, in which cationic coagulates such as metallic ions and organic polymer are added to improve dewaterability, and the centrate is returned to the mainstream (Burton et al., 2013). Due to the anionic nature of melanoidins, coagulation with various metallic ions have been demonstrated as an effective way for melanoidin precipitation (Gomyo and Horikoshi, 1976). Dwyer et al. (2009) reported simultaneous color and DON removal with alum addition into the melanoidin-containing sewage treatment plant effluent. The results showed that an alum dose of 30 mg Al L −1 was able to achieve the color, DON, and DOC removal of 75%, 42%, and 30%, respectively. In another study, adding cationic organic polymer was reported to remove 55% of DON and 45% of UV-quenching in the sludge THP return liquor after dewatering, and doing so was particularly effective for removing the DON species with molecular weights above 3 kDa which have a higher potential to bind with cationic polymer (Ahuja et al., 2016). Effective DON and UV-quenching mitigation were also reported by using dual conditioning with cationic polymer and ferric chloride during dewatering (Wilson et al., 2011).

Other post-THP treatment technologies
Melanoidins removal by adsorption has been investigated and showed good removal performance. Penaud et al. (2000) studied the color removal effectiveness of adsorbent resin (Amberlite XAD 7HP from Rhom and Haas) in the sludge THP effluent and achieved a maximum color removal of 58%. The molecular weight analysis further revealed that melanoidins with molecular weights higher than 100 kDa were eliminated, but the ones with relatively low molecular weights of 9-82 kDa were only partially removed. Activated carbon adsorption of melanoidins was also investigated using isotherm approach and the best melanoidin adsorption capacity was 450 ± 10 mg COD g −1 (Figaro et al., 2006).
Advanced oxidation processes have also been tested for decolorization of melanoidins prepared from model system. Hayase et al. (1984) reported about 64% and 97% color reductions of melanoidins using hydrogen peroxide at pH of 7 and 10, respectively. Kim et al. (1985) reported 84% and 97% removals of melanoidins by ozone treatment for 10 min and 90 min, respectively. Dwyer et al. (2008a) reported that the UV irradiation of hydrogen peroxide was capable of removing 99% color, 50% DOC, and 25% DON from the synthetic melanoidin solution at the hydrogen peroxide dose of 3.3 g L −1 .
Several microorganisms have been identified with the ability to utilize melanoidins even though they are well-known to be recalcitrant and even toxic to living organisms. Tiwari et al. (2012) reported that yeast Candida tropicalis RG-9 was able to degrade melanoidins and showed a maximum decolonization rate of 75% within 24 h when incubated at 45°C with additional carbon (glucose) and nitrogen (peptone) added. In addition, several white-rot fungus species also showed decolonization capacity of melanoidins (Dahiya et al., 2001, Raghukumar et al., 2004, Raghukumar and Rivonkar, 2001. A comprehensive review on microbial and enzymatic degradation of melanoidins was published by Chandra et al. (2008).

Conclusions
The following concluding remarks can be drawn from this review: 1. Maillard reactions are a group of browning reactions between sugar and amino group at elevated temperature and responsible for the formation of heterogeneous pigments known as melanoidins. The Maillard reaction occurs during sludge THP and leads to increased rDON in WRRF effluent. This is because THP conditions overlap with that of the Maillard reaction, and municipal sludge is rich of potential reactants for the Maillard reaction. 2. Melanoidins are recalcitrant, nitrogen-containing, negatively charged, with color and UV-quenching ability, and can be either of high (> 10 kDa) or low (< 3.5 kDa) molecular weight. Very often, melanoidins were used as the representative compounds of rDON in THP pretreated sludge. 3. Four factors impacting rDON formation were identified in this review. First, different reactant compositions are known to affect the rDON content, color development, reaction kinetics, and reaction pathways. Second, high temperature accelerates the rate-limiting step of the Maillard reaction, namely the formation of Schiff's base complex in early-stage, hence affects the physiochemical properties of melanoidins including molecular weight, color, and the rDON content. Third, pH affects the reactivity of amino group and the structure of sugar via the equilibrium shifts of different forms of the reactants. In general, low pH mitigates rDON production. Last but not the least, metallic ions affect the formation and solubility of rDON through the mechanisms of chelation, transition metal catalyzation, and coagulation. 4. The significance of the Maillard reaction in contribution to rDON production during sludge THP needs to be also evaluated in aerobic and anoxic conditions similar to the mainstream treatment. 5. Few studies focused on the characteristics of rDON produced from sludge THP. The impacts of many operational conditions of THP such as feedstock, moisture, pressure, pH, buffer capacity, and metal addition on the production of rDON are still unknown. 6. Lowering THP temperature and pH, proper selection of coagulants for upstream treatment, and sludge conditioning prior to dewatering were proposed in this review as potential strategies to control rDON production. Approaches for post-THP rDON removal were also disscussed.