Using community surveys with participatory mapping to monitor comprehensive plan implementation

Comprehensive or general plans are long-range documents intended to guide future urban or regional land use, growth, and development. Structured and periodic monitoring and evaluation of plan implementation is important to identifying when plans should be revised or updated based on changed planning assumptions or conditions, but such monitoring is uncommon. In this study we present and illustrate a research-based method to evaluate general plan implementation for a case-study community located in central California. A community survey was combined with participatory mapping to assess continued public approval of key elements of the general plan: 1) residential growth, 2) community development needs, 3) preferred locations for development (spatial), 4) consistency of resident land use preferences with general plan categories (spatial)


Introduction
Comprehensive plans, commonly called general plans, are the primary means by which local governments identify and implement longrange development goals and objectives.Comprehensive plans identify the geographic area that is subject to planning and regulatory jurisdiction, include most if not all matters related to physical development of the community, and cover a long-time horizon, typically 20 years (Kelly, 2012).A comprehensive plan should describe existing conditions, identify goals and objectives, contain plan implementation strategies, and present a future land-use map for the community.The core elements of a comprehensive plan typically contain information about community demographics, identification of socially and culturally important places, an inventory of economic conditions, a projection of future housing needs, an analysis of public facilities and services, and a description of environmental conditions, including parks and open space.While the sequence of tasks can vary, the planning process begins with an analysis of existing conditions and community visioning (i.e., goals and objectives) and is followed by plan development; the process concludes with public hearings and formal plan adoption.Public and stakeholder input into the plan can take various forms including community surveys, focus groups, and open houses.
Given the long-range planning horizon where local conditions can change, monitoring and evaluation of comprehensive plan implementation is important but often overlooked and underutilized.By contrast, many state and federal land management agencies are expected to routinely monitor the efficacy of, and adherence to, plan elements.Although evaluation of plan implementation has gained increased attention in Europe, the United States, and other regions over the last two decades, studies consistently emphasize that implementation evaluation in planning is less developed than in other fields of policy science (Lyles et al., 2016).In comparison with the attention paid to the evaluation of the quality of plans as documents, implementation evaluation is underdeveloped.Consequently, there are few methodologies for systematically evaluating plan implementation, especially methods that combine quantitative and qualitative approaches and that include community participation (Kinzer, 2016).To address these gaps, in this study we present and illustrate a research-based method to evaluate general plan implementation, focusing on the residential development analysis and general plan consistency and conflict analysis.The online participatory mapping approach applied in the current study potentially provides useful location-specific knowledge about the successfulness of the plan implementation.
Numerous reasons exist why systematic monitoring and evaluation of urban plan implementation does not occur, such as disagreement on when plan results should be determined, what criteria should be used to measure planning effectiveness, the difficulty of assessing planning impacts over long periods of time, and ambiguity in the concept of success in planning (Brody and Highfield, 2005).Pragmatically, urban planning departments, like other public sectors, face resource constraints (time, money, and skills) to monitor and evaluate plans situated in organizations that appear preoccupied with facilitating development activities (Seasons, 2003); moreover, these departments often face intense political pressure.

Conformance and performance-based plan implementation evaluation
Plan implementation evaluation can be carried out during three phases of the planning process: (1) ex ante evaluations occur during plan preparation when one solution path is chosen from among alternative plan-proposals; (2) ongoing evaluation takes place during plan implementation; and (3) ex post evaluations occur after the plan is implemented.The planning profession has mainly focused on ex ante evaluations, neglecting ongoing and ex post evaluations (Guyadeen & Seasons, 2016).An evaluation of the plan near the end of the planning cycle is essentially an ex post evaluation and should be reasonably thorough and use multiple criteria to inform the next comprehensive plan.
There are two primary approaches to ex post plan evaluation-conformance and performance-based (Berke et al., 2006;Oliveira and Pinho, 2010)-that reflect the rational and communicative approaches to plan evaluation (Guyadeen & Seasons, 2016).A conformance-based approach judges the success (or failure) of planning based on the conformance between outcomes on the ground and the plan proposals (Alexander, 2006).A performance-based evaluation judges plan effectiveness by whether, how, and under what conditions a plan effectively guides planning decisions.
As observed by Faludi (2000), to the extent that rational, comprehensive plans are strategic, requiring the coordination of development projects by a multitude of actors, a performance-based approach to evaluation is applicable.However, given that there are several recognized models of planning, comprehensive plans are not necessarily purely strategic, and they contain goals and policies that can be assessed for conformance, such as the rate of residential growth.Comprehensive plans are also spatial, with designated areas for various types of preferred development that can be evaluated against actual development outcomes on the ground.Thus, comprehensive plan monitoring and evaluation may contain measures that can be described as assessing both conformance and performance outcomes (Feitelson et al., 2017).Monitoring is typically a routine and limited activity with frequent intervals of data collection focused on a subset of plan activities and outputs.In contrast, plan evaluation is typically done across longer time intervals with more extensive data collection to address broader planning outcomes and impacts.Given that comprehensive plans have a long planning horizon-often 20 years-an assessment of comprehensive plan implementation at five-year intervals can be both a monitoring activity and a periodic evaluation, depending on the depth of assessment.
For example, if affordable housing is identified in a comprehensive plan as a high priority need but is approved in an area designated for commercial development, the amended land use would be technically non-conforming according to the original plan.This action could, however, be interpreted as a positive performance outcome because the plan identified affordable housing as a priority.New and spatially more flexible conformance analysis has been developed to alleviate the oversimplification and rigidity of traditional conformance analysis (Shen et al., 2020).Moreover, the need for detailed zoning regulations have been challenged and rule-based urban codes have been suggested as replacement for traditional statutory land use plans (Alfasi et al., 2012).
In contrast to the single comprehensive ex post evaluation of the outcomes of the plan's implementation, monitoring and evaluation should be conducted at periodic intervals.In addition, monitoring should focus on key indicators because the purpose is to determine whether local conditions have changed such that they are still consistent (or not) with community expectations and especially whether performance standard elements have been violated.Important inquiries include which plan elements should be monitored, how frequently, and using what methods, as well as what actions, if any, should be taken, given the results.In early comprehensive plan monitoring (e.g., after five years), there will be less experience and information to assess longterm plan implementation (however, should changes occur within a 5year period, there may be enough evidence such that it may be unwise to wait for a prescribed period of monitoring).Development decisions will likely have been made, but not necessarily physically implemented on the ground.
As a starting point, we suggest that comprehensive plan monitoring design should: 1) focus on the most important planning and development issue(s) for the community, revealed e.g., through community surveys; 2) include community participation in monitoring because residents are the target of plan implementation; 3) use measures and indicators from baseline data (if available) such as community surveys that can provide longitudinal information about changed conditions; and 4) include spatial measures because plan implementation is inherently place-based.In referring to ex post evaluation, Talen (1996) argued that planning requires a distinctive brand of evaluation research that accounts for the physical, spatial side of planning.This principle applies equally to comprehensive plan monitoring where it is useful to examine the spatial relationship between general land uses identified in the plan and the spatial locations of land use decisions after plan adoption.

Participatory mapping as a tool for plan implementation evaluation
With participatory mapping it is possible to combine appreciative G. Brown et al. knowledge from participants with spatially specific information about physical planning solutions (Kyttä, 2011).Thus, participatory mapping can be a potential tool for plan evaluation, helping identify place-based conflicts among various interests and finding optimized community land use outcomes.Over the past two decades, participatory mapping has emerged as a valuable method for multiple planning applications, including urban and regional planning (Brown and Kyttä, 2014;Kahila-Tani et al., 2019).Participatory mapping is a general term describing a range of participatory spatial methods that include "public participation GIS," (PPGIS), "participatory GIS" (PGIS), and some types of "volunteered geographic information" (VGI) systems.Such terms are often used interchangeably in the public and planning lexicons, but all seek to engage broader segments of the public in the planning process through the collection of spatial information (Brown and Kyttä, 2018) and to allow additional opportunities for those unable to participate in traditional planning meetings for one reason or another.
Participatory mapping has been most frequently applied in the early stages of comprehensive, general, or master plan development (see e.g., Kahila-Tani et al., 2016;Brown et al., 2018), but the method can also be used to evaluate plan implementation.Kahila-Tani et al. (2019) reported over 200 participatory planning cases from 10 countries, where online participatory mapping has been used.Thirty-seven percent of these cases focused on the evaluation phase.Participatory mapping has been used to monitor development preferences in the context of a sustainable tourism management system (Brown and Weber, 2013) and in ex post plan evaluation measuring urban environmental quality based on the relationship between urban structure and experiential, behavioral, and health variables of residents (Kyttä, 2011).However, there are few published cases of participatory mapping describing how the approach can be used for the systematic monitoring and evaluation of urban or regional plans and zoning.
In this study, we present a method for monitoring and evaluating comprehensive plan implementation that uses participatory (PPGIS) mapping as a key component.Because participatory mapping can be combined with a conventional community survey, the method has the capacity to provide important spatial and non-spatial indicators of community perceptions regarding land uses that appear consistent/ inconsistent with comprehensive plan designations.We call this consistency analysis, and it is described below.The spatial data also provides for conflict analysis, a method that identifies areas where residents agree/disagree about preferred land uses.Although comprehensive plan monitoring and evaluation are necessarily specific to the community comprehensive planning context, the methods illustrated herein can be generalized to other urban and regional plans.

Comprehensive plan monitoring location and context
We illustrate participatory mapping as a monitoring and evaluation method for San Luis Obispo (SLO), a small city located in the central coast region of California, U.S. The city has an estimated population of 46,724 (SLOCOG, 2017) with a perceived high quality of life, ranking 17th best place to live among U.S. cities in 2018 (livability.com, n.d.).The city has a favorable climate, abundant natural attractions for outdoor activities, and a historic downtown that supports tourism; it is also home to California Polytechnic State University (Cal Poly), a highly ranked regional university.Along with these attributes come significant pressure for community growth, but housing in the San Luis Obispo area is among the least affordable in the U.S. based on median family income, ranking seventh least affordable of 237 metropolitan areas in (National Association of Home Builders, 2019).As a result, the dominant local planning issue is the perceived need for affordable housing, also called "workforce" housing.The city formally adopted its comprehensive plan (called General Plan or "SLO 2035) on December 9, (City of San Luis Obispo, 2019), see Fig. 1.This was preceded by a G. Brown et al. general plan revision in 2012.In terms of residential growth, the projected number of residential dwellings would increase from 20,697 in 2013 to 25,762 in 2035 under the stated policy of not exceeding 1% growth per year on average.The five-year plan monitoring interval covered by this study (2014)(2015)(2016)(2017)(2018)(2019) called for up to 1,144 new dwelling units to meet regional housing needs allocation (General Plan, Housing Element).In 2014, a total of 902 units were built, approved, or under construction and were credited toward the 1,144-unit target, or about 60% of the five-year goal.
After adoption of the San Luis Obispo general plan and within the five-year evaluation timeframe, three large, mixed-use development projects were approved, raising community concern about whether these projects were consistent with general plan goals and policies (Brown and Eckold, 2020).These developments appear in Fig. 1 in specific planning areas called SP-2 (San Luis Ranch), SP-3 (Madonna on LOVR), and SP-4 (Avila Ranch).Two of these development projects were formally approved by the SLO City Council in 2017 as planned-unit developments.The Avila Ranch development proposed 720 residential units (estimated 1,649 people) with commercial space on a 150-acre site used for agriculture.The San Luis Ranch development project proposed 520 units (282 single-family and 298 multi-family) with commercial space on a 131-acre site.A third development, Madonna on LOVR (henceforth, Froom Ranch), proposed a mixed-use senior housing project on 110 acres with 350 residential units for seniors, 200 apartments, around 60-100 single-family detached units, and commercial space.When fully developed, the three projects would add over 1,800 units to existing housing inventory in the city, or some 650 units more than planned for by 2035.

Plan monitoring and evaluation design
As part as of a university-sponsored research project, we designed a participatory mapping method for monitoring and evaluation of comprehensive plan covering the five-year interval (2014-2019) Fig. 2. The participatory mapping approach used in this study has been widely used in other contexts, though specific details often vary with circumstances such as issues, target population characteristics, cost, and political backing or required permissions.The approach used in this study is consistent with the above considerations.
In 2012, a community survey was conducted in SLO as part of the general plan revision; it provided baseline information to assess changes in community opinions about the types and locations of growth and development.Participatory mapping had not previously been conducted in SLO.Key indicators selected for monitoring measurement were residential growth rate and spatial preferences for development, the adequacy of various types of development, and community support for major development decisions with significant potential impact on general plan goals and policies (Table 1).These indicators were selected to provide information about general plan implementation to date, as well as to identify future plan implementation issues.
With digital markers for eight defined community place values (panel 1) and 10 preferences favoring various land uses such as residential development (panel 2) or opposing the same land use (panel 3).The method described herein focuses on development preferences.Participants were instructed to drag and drop markers onto the map indicating locations where they preferred (or did not prefer) to see specific land uses.The number of markers were not limited.Of particular interest were the project locations for three large developments that were approved after adoption of the 2014 General Plan update.
The survey included a set of community planning questions that asked participants about preferences for the rate of annual housing growth based on the SLO general plan limit of 1% (decrease, no change, increase); the preferred location for future residential development; the adequacy of current levels of various types of development including residential, commercial, tourism, and manufacturing; and participants' agreement/disagreement with the three large development projects previously described.To assess participant representation, the survey also contained questions about participants' age, gender, education, employment status, length of residence, and whether they rent or own a residence.
Study participants were recruited from two sources: 1) a random household sample in SLO, and 2) a volunteer (convenience) sample of participants from social media sources and a university digital newsletter.To select the household sample, a list of 10,000 names and addresses of households in SLO was purchased from a commercial vendor.From this list, 1,595 households were randomly selected, geocoded, and reviewed to ensure households were geographically distributed across all city areas.Volunteers were recruited from a social media website called "Nextdoor Neighbor," which has approximately 6,500 subscribers, and from a posting in the California State Polytechnic University weekly digital newsletter that is distributed to university faculty, staff, and students.
The survey recruitment letter sent to randomly selected households identified the purpose of the study and provided an internet URL with a unique access code to allow the tracking of responses.After two weeks, a reminder postcard was sent to non-respondents followed by a second postcard reminder at four weeks.Volunteers were also directed to the same survey website, but responses from the two sampling groups (household, convenience) were tracked separately.Survey responses were collected from March through April 2019.

Analysis of survey responses
To assess sample representativeness and potential bias, we compared survey respondents with census data for SLO on the variables of age, gender, and level of formal education.Other demographic variables collected included employment status, housing (rent/own), and years of residence.Survey question responses were analyzed by sampling group (household vs. volunteer) as well as pooled responses.We used z-or tstatistical tests to examine significant differences between the household and volunteer samples depending on the survey question and variable type (scale or categorical).

Residential development analysis
With a special focus on residential development, we analyzed the mapped residential development preferences (supporting/opposing) for single-family, multi-family, and affordable housing categories.These preferences were counted for the planning area (city limits) and for subareas within the city (i.e., central, northeast, south, southeast, and southwest) to compare with survey results and general plan policies.
Residential development preferences were also counted within three large development projects with significant influence on the residential growth goals identified in the general plan.

General plan consistency analysis
General plan consistency analysis examines whether the mapped distribution of land use preferences appear logically consistent with the general plan's land use categories.For example, mapped preferences supporting residential development would be logically consistent if mapped in the general plan categories of single-family, multi-family, and affordable housing but inconsistent if mapped in other general plan categories such as open space or public park land.To conduct the analysis, we combined mapping data for all participants and crosstabulated the frequency of mapped land use preferences (supporting/ opposing) by general plan categories using chi-square statistics and standardized residuals.The standardized residuals indicate which types of land use preferences appear to be significantly under or overrepresented in the general plan land use category.Residual values greater than + 2.0 indicate significantly more preference markers than expected by general plan category, while residual values less than −2.0 indicate significantly fewer markers than expected.

General plan conflict analysis
There are multiple ways to assess land use conflict potential using participatory mapping data.The calculation of multiple conflict indices is presented in Brown and Raymond (2014) and Karimi and Brown (2017).For this study, we used the weighted preference score (WPS) index which measures the degree of mapping agreement between supporting and opposing preferences for the same land use in the same geographic location.The WPS index is computed as a ratio that varies between 0 (highest level of agreement where all preferences either support or oppose the land use) and 1 (lowest level of agreement where preferences are evenly divided between supporting and opposing preferences).The calculated ratio is then weighted by the number of mapped preferences in the geographic area.We used a sampling grid approach (fishnet) overlaying the planning area (city limits with 1 km buffer) to calculate conflict index values per cell.The best grid-cell size is an analyst judgement based on the quantity of mapped data and the spatial scale appropriate for assessing land use options.We determined the best size would be a 200-meter grid cell based on the quantity of spatial data and the size of the planning area.This generated a total of 1,940 grid cells for the study area.For our analysis, we focused on the conflict potential for residential development-the key land use issue for the city.The conflict index for each grid cell was calculated based on the ratio of summed preferences in support of single-family, multi-family, and affordable housing with summed preferences opposing these land uses.For comparison, we overlaid the grid-based conflict scores on the general plan map to highlight general plan designations where residents appear most divided about future development.

Survey response and participant characteristics
A total of 221 individuals mapped one or more locations with 111 random household sample participants and 110 volunteer participants (see Table 2).A total of 144 participants completed the post-mapping survey questions.The household survey response rate was 9.2% after accounting for non-deliverable recruitment letters.There were 8,370 mapped locations, with the average number of markers equal to 38 and a median value of 18.On average, household sample participants mapped more locations than volunteers did; this finding is consistent with other participatory mapping studies (Brown, 2017).
The household sample had a significantly greater proportion of retirees than the volunteer sample (29% vs. 10%), while the volunteer sample had a significantly greater proportion of college students (22% (the proportion of those over 65 years old) and 19% (the proportion of college students of the total labor force).The household sample had lived in the community longer (23 vs. 14 years), was significantly older (56 vs. 41 years), held a higher level of formal education (85% vs. 68% with a college degree), and represented greater home ownership (89% own vs. 53% rent).The volunteer sample had significantly more female participants (67% vs. 44%).When compared with census estimates for the population of SLO, the pooled sample of participants was older and more formally educated, and the sample contained a greater number of females.Direct comparisons of the sample population characteristics with Census Bureau statistics for SLO are complicated because the simpler and more direct categories used for the purposes of the survey do not necessarily coincide with the census categories.

Residential growth and development
The SLO general plan calls for a maximum annual average residential growth rate of 1%.Residents were asked if this desired rate of growth should stay the same, increase, or decrease.The majority of participants (56%) responded that the growth rate should stay the same, while 36% said the growth rate should be less (Table 2).Only 9% said the rate should be increased.These monitoring results are nearly identical to community survey results from 2012 (City of San Luis Obispo, 2012), where about 55% of respondents said there should be no change in residential growth rate.
One survey question asked participants about the preferred area/ preferred residential locations for new residential development within SLO; these preferred areas were in the south (23% / 34%, preferred area/preferred residential location, respectively), central (21% / 20%), southeast (20% / 25%), northwest (18% / 14%), and southwest (11% / 7%).Although the general plan does not establish quantitative goals for residential development by area, it does contain a residential capacity inventory (General Plan, Appendix K), allowing assessment of whether community preferences appear aligned with residential capacity.The estimated residential capacities by area in the general plan were south (20%), central (19%), southeast (38%), northwest (5%), and southwest (17%); thus, there is a rather high agreement between city residential capacity and resident preferences that prioritize residential development in the south and southeast areas as opposed to the southwest and northwest areas.
A survey question asked participants about the adequacy of the current level of land uses, including different types of residential development.Response categories were "enough", "not enough", or "too much".Two residential categories, affordable housing (79%) and multifamily housing (55%), were perceived as "not enough" (see Fig. 3).All other land uses, with the exception of bicycle paths, were perceived as adequate ("enough").Although the survey questions were not identical, these results are consistent with results from the 2012 community survey conducted for the 2014 General Plan update.
The survey asked whether participants agreed/disagreed with three large, mixed-use development projects, two of which were approved in 2017 (Avila Ranch, San Luis Ranch) and one that was pending final approval (Froom Ranch), after adoption of the General Plan in 2014.Project agreement was also measured by examining the number of supporting or opposing development preference markers located within the three project sites.Survey responses for all three projects were almost equally divided between agreement/disagreement (see Table 3) with up to 20% of respondents yet unfamiliar with the projects.Responses for Avila Ranch were divided 42% in agreement and 42% in disagreement, while responses for San Luis Ranch were divided 40% in agreement and 44% in disagreement.The mapping results also revealed deep community ambivalence about the development projects: The mapping results for Avila Ranch and San Luis Ranch indicated contrasting outcomes with mapped preferences supporting the Avila Ranch development by a margin of 50% to 33% and mapped preferences opposing San Luis Ranch by a margin of 45% to 26%.Some of these differences may be explained by fewer participants who mapped preferences in the project areas compared to the larger number of responses from the survey question.

General plan consistency analysis.
We examined whether the distribution of mapped preferences by general plan category were logically consistent using chi-square/ G. Brown et al. residuals analysis. 1The analysis was done for supporting and opposing land use preferences.For supporting land use preferences, there was an overall moderate association between preferences and general plan classification (Х 2 = 1471.9,df = 120, p < 0.001; Cramer's V = 0.26, p < 0.001); see Table 4.The majority of pair-wise chi-square residuals showed logical consistency between the preferred land use and the general plan category.For example, preferences supporting commercial/retail were mapped significantly more than expected in the general plan categories of general retail (residual=+8.3)and community commercial (+3.9), manufacturing preferences were mapped significantly more in the manufacturing category (+15.5), and open space preferences were mapped more in the open space and public park land categories (+18.2).Preferences for single-family homes were mapped significantly more in the low-density housing category (+12.2, while tourism-serving preferences were mapped more in the general tourist commercial (+5.6) and commercial retail (+6.7) categories.
There was one inconsistent finding indicative of a mismatch between general plan category and preferred land use.There was greater frequency of single-family housing preferences placed in the Business park category than would be expected (+4.3).A closer examination of the spatial data revealed that a large majority of these residential preferences were located in one of the six areas designated in the general plan as Business park; this area may warrant further investigation for potential general plan amendment, for example, to gain insight into whether preferences for existing business parks or proposed housing were anomalous.
For opposing land use preferences, there was a moderate, but weaker association between the distribution of preferences by general plan classification (Х 2 = 350.3,df = 56, p < 0.001; Cramer's V = 0.19, p < 0.001) with fewer significant pair-wise associations than with supporting land uses.See Table 5.There were significantly more "no bicycle path" markers mapped in residential categories than would be expected and significantly fewer "no development" markers mapped in the general plan categories for residential use.These findings suggest that participants oppose additional bicycle paths in residential areas but are more open to new development occurring in these areas.There were significantly more "no development" markers in the open-space and public parks categories, a confirmation of the importance of open space to residents found in text-based survey questions.Fig. 3. Adequacy of current level of various types of general plan land uses in San Luis Obispo.All land uses were perceived as adequate with two exceptions: affordable housing with 79% indicating "not enough" and multi-family housing with 55% indicating not enough.

Table 3
Results of public opinion for three large development projects (Avila Ranch, San Luis Ranch, and Froom Ranch) from two different information sources: a text-based survey and counts of mapping preference markers located in the project area.Note: Overall association is statistically significant (Х 2 = 1471.9,df = 120, p < 0.001; Cramer's V = 0.26, p < 0.001) with statistically significant residuals indicating over-representation (> +2.0) or under-representation of preferences (<−.2.0) within the general plan classification.
G. Brown et al.

General plan conflict analysis
To examine conflict potential for residential development with the general plan, we first summed and plotted the number of preferences supporting residential development in 200 m grid cells overlaying the planning area.The number of preferences ranged from 0 to 20 markers per cell, and a density map of preferences by grid cell appears in Fig. 4a.The same procedure was followed for preferences opposing residential development.The number of preferences ranged from 0 to 16, and the density map appears in Fig. 4b.For each grid cell, the conflict index (weighted preference score or WPS) was calculated as the ratio of supporting preferences to opposing preferences (range 0 to 1) multiplied by the total number of preferences in the grid cell.Index values ranged from 0 (no preferences) to 16.The conflict index map appears in Fig. 4c and  4d.Areas with the greatest potential for conflict include the three special planning areas (Avila Ranch, San Luis Ranch, and Froom Ranch), a currently undeveloped area in the southeast designated for residential land use, the south-central area designated for residential, and a parcel of land in the northwest designated for public facilities.With the exception of the northwest location, the highest conflict potential locations are currently undeveloped but designated for residential development in the general plan-perhaps an indication of future land use conflict if left unexamined.

Discussion
In this study, online participatory mapping was used to evaluate general plan implementation.The case study in San Luis Obispo of central California showed that participatory mapping can be used as a systematic, ex post evaluation method for plan implementation that includes community participation.This methodology thus contributes to some essential methodological deficiencies of planning evaluation scholarship identified by Lyles et al. (2016), Guyadeen and Seasons (2016) and Kinzer (2016).Furthermore, the place-based approach of participatory mapping methodology makes it possible to simultaneously assess both conformance and performance outcomes (Feitelson et al., 2017).This was possible by applying place-specific residential development analysis and general plan consistency and conflict analysis to study the selected key indicators for monitoring.
Residential growth rate and spatial preferences for development were studied by analyzing resident acceptability of the general plan principles.The stated preferences of participants for the rate of annual housing growth revealed that the average annual residential growth rate of 1% still holds five years into the 20-year plan, as does the perceived adequacy of various types of other development such as commercial/retail.The city planning department produces a general plan annual report that is intended to evaluate whether "actions that have occurred indicate a change in the general vision of the community that requires a more comprehensive update of the General Plan" (City of San Luis Obispo, 2018).This is an important part of general plan monitoring that includes a quantitative assessment of development applications and approvals.However, in the absence of a community survey or other form of public engagement to directly measure community perception, one cannot reliably assess a "change in the general vision of the community" that may arise with changing demographic and economic characteristics and values of the population.
The locations of growth and development and the adequacy of various types of development were studied with general plan consistency and conflict analysis, respectively.The three large development projects that were approved after general plan adoption are in conformance with the Note: Overall association is statistically significant (Х 2 = 350.3,df = 56, p < 0.001; Cramer's V = 0.19, p < 0.001) with statistically significant residuals indicating over-representation (> +2.0) or under-representation of preferences (<−.2.0) within the general plan classification.
plan because the plan identified these special planning areas as places where a planned mixed-use development would be appropriate.However, the details of the size and scale of the developments, as well as the proportional mix of land uses, were not provided in the general plan, potentially setting up contradictory or conflicting directions.It is only when the details of a proposed mixed-use planned unit development become public that residents can meaningfully evaluate the costs and benefits of the development and consistency with general plan policies.
In other words, a proposed development type can conform to designated land use in the general plan but still be inconsistent in detail with other plan policies.This explains why some residents expressed concerns about the scale of the projects and the prima facie contradiction with other plan policies such as open space protection, gradual growth to maintain a compact urban form, and protection of air quality (Brown and Eckold, 2020).Community division over the three development projects found in both survey and mapping results indicates that residents are conflicted about issues regarding preservation of open space, the most important community value identified in the general plan, and new development that could potentially reduce the cost of housing.An argument can be made that approval of 1,800 housing units is inconsistent with general plan constraints on residential growth (1% annually) when the general plan called for adding about 1,200 units over a five-year period (2014-2019).However, these three projects will be constructed in phases such that the average annual growth rate can remain in compliance with general plan constraints.In this case, the spatial location of new development may be more important than the actual number of housing units.The community impacts of new development projects (positive and negative) in existing open space may be more obvious (but not necessarily less contentious) than smaller infill or land redevelopment projects.
The kind of systematic monitoring and evaluation of urban or regional plans and zoning realized in this case study in San Luis Obispo represents an example of the infrequent use of a participatory mapping method in evaluating plan implementation.The method has most often been used in the early stages of the planning process (Kahila-Tani et al. 2016;Brown et al., 2018).The challenges related to the use of participatory mapping method are similar regardless of the planning phase when this method has been used.
A primary concern with participatory mapping, and with community surveys in general, is the response and/or volunteer participation rates which affect community representativeness of the results.The household sample response rate was about 10%, less than desired but not atypical for many social surveys in recent years.The household response was biased toward older, male, highly educated, long-term residents, with a higher proportion of home ownership.Some of this bias was partially offset by the volunteer sample that included younger, female, shorter-term residents who rent, and many of whom attend college.In addition to demographic bias, participatory mapping is sensitive to the quantity of participants and resulting mapped data.While over 200 participants and 3,600 mapped preferences may appear adequate for a monitoring effort, the distribution of markers will be spatially clustered, leaving portions of the planning area without adequate data to draw strong inferences about plan consistency or conflict potential difficult.
The number of participants and the quantity of spatial data becomes important if the data are to be used to justify the need for general plan revision.
Another potential limitation is the level of trust necessary to achieve broad community participation, a combination of familiarity with the participatory mapping method, the perceived legitimacy of the assessment sponsors, and what socioeconomic or political interests of different segments of the population may be at stake.Many residents are for various reasons naturally skeptical of social data collection, with one key concern being whether and how the information will be used.Although most residents will have had experience with general survey research, few have experience with participatory mapping, though the technique is becoming more widely used.Participatory mapping requires a level of participant engagement (time and effort) that exceeds traditional text-based survey questions; this partially explains lower overall response rates.Further, participatory mapping is influenced by the expectation that the information will be used in decisions that affect the participant.Thus, local government sponsorship of monitoring, and expectation that public contributions will actually influence decision making, is an important contributor to participant perceived efficacy.Still, the data collection method must be viewed as objective.In the case of the SLO study community, the evaluation was implemented as a university-sponsored research project, a positive for study objectivity but a negative for participant perceived efficacy because it lacked official local government sponsorship.An ideal combination would be local government sponsorship implemented through unbiased, in-depth, and outcome-oriented university research.To overcome natural tendencies for low response rates in public surveys, higher response rates in participatory mapping (up to 39%) are possible with data collection strategies that emphasize personal invitations (Kyttä, 2011;Kahila-Tani et al., 2019).
A final methodological consideration is the choice of spatial markers.We chose preference markers representing different types of development to assess plan consistency.However, there is a tendency for participants to view developed land as difficult to change, so participants are more likely to express preferences for undeveloped land.To overcome this bias toward the status quo and to encourage participants to think more openly about the type and location of appropriate future development, markers could be included that explicitly identify areas for changes in performance zoning or redevelopment.
Some of these limitations to this application of participatory mapping are simply extensions of those in contemporary community planning.For example, neither can ensure that results perfectly represent the values and will of populations, although well-conducted survey research may certainly achieve more testable results.Issues of scale are problematic for both, though the approach taken in this study may be more precise than traditional non-specific commenting.In this study, one clear issue with the PPGIS survey was that some individuals used the possibility to pinpoint a disproportionally large number of place markings, which reduces reliability and usefulness of findings.We recommend that for future studies, the maximum number of mappings that an individual survey respondent is able to mark are restricted.
When interpreting the findings, it is important to acknowledge compromises related to the representativeness, sample size, and quality of PPGIS data.When these limitations have been considered, there is no reason why the results of this study could not advise the city planners of San Luis Obispo.The purpose of the current study was to showcase the possibilities of a simple online survey to provide knowledge for the monitoring and evaluating of comprehensive plan implementation.Without the location-specific approach introduced here, the consistency and conflict analysis would be difficult to realize.Other cities can be inspired to try to collect larger datasets, for example via more efficient marketing and with promises that the produced information will be taken seriously and used in decisions that affect the local inhabitants.

Conclusion
In this study, we demonstrated how participatory mapping can be used to monitor and evaluate general plan implementation from a community perspective.A general plan map presents the spatial future for a community by showing the intended locations of various types of development, as well as areas for parks/open space.Participatory mapping provides an opportunity to conduct consistency analysis, which examines whether community preferences are consistent with general plan land-use categories, and conflict analysis, which identifies locations where residents hold different opinions about the appropriate type or level of development.There are few robust methodologies for systematically evaluating plan implementation and even fewer that incorporate a participatory spatial component.An evaluation approach with quantitative assessment of development (e.g., growth rate, development approvals) is important and necessary, but the importance of "place" decisions to residents is typically not evaluated.The "where" of plan implementation appears equally important to the "how much" in achieving the community vision.The type and quality of development are also important but are more difficult to assess without specific development proposals or performance zoning details.Comprehensive plans will invariably contain goal ambiguity, if not conflict, and rarely explicitly rank or prioritize goals or acknowledge tradeoffs in the face of conflicting socioeconomic values and the ever-present preference for purposefully using ambiguous terms.Spatial preferences provide an empirical basis for evaluating goal priorities and tradeoffs.Arguably, plan monitoring and evaluation, as widely used in state and federal government land management planning, should provide more than conformance assessment but also the capacity to assess whether development decisions, even if nominally conforming, are consistent with community values and preferences.A community survey with participatory mapping not only provides a means to assess consistency with plan assumptions but can proactively identify the potential for community conflict to find optimizing development outcomes and need for change.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Fig. 1 .
Fig. 1.San Luis Obispo land use map from the General Plan showing designated land uses.

Fig. 2 .
Fig. 2. Participatory mapping interface showing two-tab panels of preference markers to be dragged and dropped onto a digital map of the study area: (a) supporting land uses; (b) opposing land uses; and (c) visual example of markers placed on the Google Map.

Fig. 4 .
Fig. 4. Maps showing: (a) number of markers supporting residential development; (b) number of markers opposing residential development; (c) conflict potential; and (d) conflict potential with selected general plan classifications.

Table 1
Selected indicators for general plan evaluation with rationale for selection.

Table 2
Respondent profile by sampling group (household and volunteer) and responses to survey questions about residential growth.Selected census data are provided for comparison.Not all percentages total 100% due to rounding.
2Mean difference between household and volunteer samples is statistically significant (t-test, p < 0.05).
Distribution of supporting land use preferences by general plan classification.
1 Residual analysis is used to determine what categories (cells) were major contributors to rejecting the null hypothesis.When the absolute value of the residual (R) is greater than 2.00, the researcher can conclude it was a major influence on a significant chi-square test statistic.See http://www.acastat.com/statbook/chisqresid.htmTable 4

Table 5
Distribution of opposing land use preferences by general plan classification.