American Woodcock Singing-ground Survey The Logistical Challenges Associated with Route Consistency through Time

The American Woodcock Singing-ground Survey (SGS) is a long-term roadside survey (1968–present) administered by the U.S. Fish and Wildlife Service (USFWS). The SGS was developed to provide indices to changes in American woodcock (Scolopax minor) abundance. The population index derived from the survey is the primary metric used for the United States (US) harvest strategy. Integral to any long-term wildlife-monitoring program (e.g., SGS) with replicated spatial point-count locations is accurate management of metadata related to those locations. Technological advances over the last 20 years have resulted in large-scale coordination and logistical planning changes for the SGS, including improved communication between stakeholders and the creation of a database that houses metadata for all point-count locations. These improvements revealed weaknesses in the historical record-keeping system used for official paper route-maps that may have led to point-count location inconsistencies over time. To summarize the scope of the problem, and make corrections, we compared submitted GPS coordinates for count locations on SGS routes against indicated route paths on official route maps. Across the entire SGS coverage area, we found that 9.9% of observer-submitted point-count coordinates did not match the route path highlighted on the official route maps. We also compared a subset of digitized Minnesota and Wisconsin submitted point-count coordinates and found that 20.9% did not match the route path highlighted on the official route map. We quantified and grouped Minnesota and Wisconsin route-map discrepancies to provide perspective on the types and magnitude of the discrepancies that occur throughout the SGS coverage area. Reasons for the mismatch were many. We share the many challenges of maintaining route consistency and provide recommendations on how to best alleviate route map discrepancies, thus improving the integrity of the SGS and its data. Proceedings of the American Woodcock Symposium 11: 217–226 Key woRdS: American woodcock, American Woodcock Singing-ground Survey, roadside survey, route consistency, route relocation, Scolopax minor, SGS, spatial data 1 email: rebecca_rau@fws.gov The American Woodcock Singing-ground Survey (SGS) occurs throughout the core of the American woodcock (Scolopax minor; hereafter woodcock) breeding range in Canada and the United States (Fig. 1, Seamans and Rau 2017). This survey began in 1968 and is administered by the U.S. Fish and Wildlife Service (USFWS) in cooperation with the Canadian Wildlife Service (CWS), state (19) and provincial (6) natural resource agencies, Bird Studies Canada (BSC), other U.S. and Canadian government organizations, and volunteer observers. The Migratory Bird Program within the USFWS is responsible for program coordination and compiling, managing, and reporting the data, whereas the United States Geological Survey (USGS) is responsible for data analysis. More than 700 natural resource agency staff and volunteer observers collect data. https://doi.org/10.24926/AWS.0128 This is a work of the U.S. federal employee and is not subject to copyright protection in the United States. Foreign copyrights may apply. 218 [5] singing-ground survey evaluation · Proceedings of the Eleventh American Woodcock Symposium The population index derived from the survey is the primary metric used for the United States (US) harvest strategy (U.S. Fish and Wildlife Service 2010). SGS data have also been used to develop population and habitat goals for the American Woodcock Conservation Plan (Kelley et al. 2008). Woodcock are managed on the basis of 2 regions, Eastern and Central, and the boundary between the 2 regions conforms to the boundary between the Atlantic and Mississippi Flyways; results from the survey are therefore designed to support management at a regional scale (Seamans and Rau 2017). Thogmartin et al. (2007) used data from this survey to develop spatially explicit models to show the relationship between woodcock abundance and landscape-level habitat variables. Field Methods An index of woodcock population abundance is estimated by taking advantage of the conspicuous breeding call of male woodcock (Seamans and Rau 2017). Early studies demonstrated that counts of singing males provide indices to woodcock population abundance and could be used to monitor annual changes (Mendall and Aldous 1943, Goudy 1960, Duke 1966, and Whitcomb 1974). Before 1968, woodcock counts were conducted on non-randomly located routes. Data from these early surveys are not used in any current analysis. In 1968, state, federal, and provincial agency personnel randomly established SGS routes along lightlytraveled secondary roads in the center of randomly-chosen 10-minute degree blocks within each state and province within the central and northern portions of the woodcock’s breeding range (Fig. 1). Each route is 5.4-km long and consists of 10 equally spaced listening point-counts (stops). The routes are surveyed once a breeding season within a designated temporal window. The SGS periods (windows) were assigned by latitude to coincide with peaks in courtship behavior of local woodcock populations (Seamans and Rau 2017). Routes are to be surveyed between 22 and 58 minutes after sunset (or, between 15 and 51 minutes after sunset on overcast evenings) by an assigned observer who drives to each of the 10 stops and records the number of woodcock heard peenting (the vocalization by displaying male woodcock on the ground). Additional guidance on survey protocols can be found within the SGS data entry website (U.S. Fish and Wildlife Service 2017). The SGS is managed by a national coordinator in the USFWS Migratory Bird Program office who works directly with 1 Canadian and 2 US regional coordinators in the 2 designated management regions (Figs. 1 and 2, Seamans and Rau 2017). Historically, the management region coordinator’s primary responsibility was to distribute survey materials to the state and provincial coordinators in their respective regions. Since the mid-2000s, the management region coordinator’s main focus has been to assist in finding state and provincial coordinators as needed and to boost participation levels in the SGS. Since the inception of the SGS, 25 state and provincial coordinators have been responsible for the distribution of survey materials, finding observers for routes, and collection and/or entry of survey data. The observers (>700) are primarily responsible for conducting the survey following standard operating procedure (SOP) guidelines (U.S. Fish and Wildlife Service 2017). Observers are encouraged to scout their routes before conducting the survey to ensure the route is safe and roads are navigable. Since 2003, observers have also been responsible for acquiring the spatial data for their route per GPS data collection standards (U.S. Fish and Wildlife Service 2017). Once GPS coordinates for a route are submitted to the national coordinator, stop coordinates are not requested again unless a route changes or there appeared to be a discrepancy. Although direct communication does occur between observers and the national coordinator, it typically only occurs when questions arise during annual validation of the survey or data. Route Map Management Historically, state and provincial coordinators have been responsible for making the official paper route maps (official route map). Map elements such as scale, content, labels and clarity were their responsibility (supporting information Fig. S1). Once established, they provided the official route map to the national coordinator who maintained them. The state and provincial coordinators retained a copy of the map, which they distributed Figure 1. American woodcock management regions, breeding range, and Singing-ground Survey coverage (Seamans and Rau 2017).

The population index derived from the survey is the primary metric used for the United States (US) harvest strategy (U.S. Fish and Wildlife Service 2010). SGS data have also been used to develop population and habitat goals for the American Woodcock Conservation Plan (Kelley et al. 2008). Woodcock are managed on the basis of 2 regions, Eastern and Central, and the boundary between the 2 regions conforms to the boundary between the Atlantic and Mississippi Flyways; results from the survey are therefore designed to support management at a regional scale (Seamans and Rau 2017). Thogmartin et al. (2007) used data from this survey to develop spatially explicit models to show the relationship between woodcock abundance and landscape-level habitat variables.

Field Methods
An index of woodcock population abundance is estimated by taking advantage of the conspicuous breeding call of male woodcock (Seamans and Rau 2017). Early studies demonstrated that counts of singing males provide indices to woodcock population abundance and could be used to monitor annual changes (Mendall and Aldous 1943, Goudy 1960, Duke 1966, and Whitcomb 1974. Before 1968, woodcock counts were conducted on non-randomly located routes. Data from these early surveys are not used in any current analysis. In 1968, state, federal, and provincial agency personnel randomly established SGS routes along lightly-traveled secondary roads in the center of randomly-chosen 10-minute degree blocks within each state and province within the central and northern portions of the woodcock's breeding range (Fig. 1). Each route is 5.4-km long and consists of 10 equally spaced listening point-counts (stops). The routes are surveyed once a breeding season within a designated temporal window. The SGS periods (windows) were assigned by latitude to coincide with peaks in courtship behavior of local woodcock populations (Seamans and Rau 2017). Routes are to be surveyed between 22 and 58 minutes after sunset (or, between 15 and 51 minutes after sunset on overcast evenings) by an assigned observer who drives to each of the 10 stops and records the number of woodcock heard peenting (the vocalization by displaying male woodcock on the ground). Additional guidance on survey protocols can be found within the SGS data entry website (U.S. Fish and Wildlife Service 2017).
The SGS is managed by a national coordinator in the USFWS Migratory Bird Program office who works directly with 1 Canadian and 2 US regional coordinators in the 2 designated management regions (Figs. 1 and 2, Seamans and Rau 2017). Historically, the management region coordinator's primary responsibility was to distribute survey materials to the state and provincial coordinators in their respective regions. Since the mid-2000s, the management region coordinator's main focus has been to assist in find-ing state and provincial coordinators as needed and to boost participation levels in the SGS. Since the inception of the SGS, 25 state and provincial coordinators have been responsible for the distribution of survey materials, finding observers for routes, and collection and/or entry of survey data. The observers (>700) are primarily responsible for conducting the survey following standard operating procedure (SOP) guidelines (U.S. Fish and Wildlife Service 2017). Observers are encouraged to scout their routes before conducting the survey to ensure the route is safe and roads are navigable. Since 2003, observers have also been responsible for acquiring the spatial data for their route per GPS data collection standards (U.S. Fish and Wildlife Service 2017). Once GPS coordinates for a route are submitted to the national coordinator, stop coordinates are not requested again unless a route changes or there appeared to be a discrepancy. Although direct communication does occur between observers and the national coordinator, it typically only occurs when questions arise during annual validation of the survey or data.

Route Map Management
Historically, state and provincial coordinators have been responsible for making the official paper route maps (official route map). Map elements such as scale, content, labels and clarity were their responsibility (supporting information Fig. S1). Once established, they provided the official route map to the national coordinator who maintained them. The state and provincial coordinators retained a copy of the map, which they distributed Figure 1. American woodcock management regions, breeding range, and Singing-ground Survey coverage (Seamans and Rau 2017). directly to both regional coordinators and observers to facilitate route sampling. In some cases, the regional coordinator may have directly distributed maps to observers. If a route problem occurred (e.g., safety to an observer, road closure), it was incumbent on the observer to notify the regional (or state/provincial) coordinator. If a route change was warranted (U.S. Fish and Wildlife Service 2017), it was the state and provincial coordinators' responsibility to ensure official route map modifications and provide the replacement map to the national coordinator and to others in the survey management hierarchy (Fig. 2).
National, state, and provincial coordinators have authorized and documented many route changes over the 50 years of the SGS. SOPs allow route changes to occur when human population density and urbanization lead to personal safety hazards such as increased volume or highspeed traffic, loss of roadside shoulders, increased crime, or other unsafe conditions. Observer safety is the primary reason why route changes have occurred. Increased noise level along routes that preclude observers from hearing woodcock peenting is another reason for route relocation. Road conditions may also necessitate a route relocation if the observer is unable to complete the route or stop during the prescribed period or peak time of daily singing activity (Seamans and Rau 2017). Although these practices have been in existence for the duration of the survey, no consistent procedural system was in place to document why a route path changed before 2001. Nor was there a common system that forced observers, state or provincial coordinators, or the national coordinator to produce an official route map that shared common features across state or provincial lines. Whereas current efforts are made to provide guidance on what to include on an updated official route map, there is no 1 sole entity that produces every map in the survey that is distributed to observers.
Currently, there is a SOP for authorized route changes when needed (U.S. Fish and Wildlife Service 2017). Once a potential new official route has been identified, there are several steps that must be completed. First, a new official route map must be produced indicating the starting location (stop 1) and ending location (stop 10), and the layout of the route is prepared by the observer or state or provincial coordinator. Second, GPS coordinates for each stop are collected by the assigned observer (U.S. Fish and Wildlife Service 2017), before surveying the new route for the first time. Third, copies of the new official route map, marking the year of establishment, along with the GPS coordinates must be sent to the national coordinator, and a copy should be retained by the state or provincial coordinator in a permanent file. The new route number will not be assigned until the new route map and GPS coordinates are received and approved by the national coordinator. Until that time, the previous route will remain in the national coordinator's database. If possible, both the original and relocated routes are to be surveyed during the year of transition by the same observer so that the data may be analyzed for short-term trend comparison (Seamans and Rau 2017). There is also an internal SOP checklist to assist the national coordinator when route changes occur. This procedure ensures new routes have maps and verified and accurate spatial data, suggested sunset times are adjusted based on the new starting coordinates, specific tables within the database are updated and associated metadata is included, new and discontinued electronic and hardcopy maps and files are updated appropriately and archived as appropriate, and communication between all parties is documented.
Undocumented route changes by observers occur, primarily due to poor communication or miscommunication between observers and coordinators, and are often discovered by chance. Insufficient map detail, staff turnover, misplaced route maps and/or observer stop descriptions, poor understanding of survey protocols, limited documentation, and the large numbers of people involved on an annual basis can contribute to these changes. Once an undocumented route change has been detected by a SGS coordinator, the problem is corrected through a set of procedures that allows the route path to revert back to its original design and/or the route number is retroactively adjusted within the database to account for the new route path.

Technological Advancements
Since the initiation of the survey in 1968, survey logistics between and among coordinators have changed considerably. Primarily driven by paper-based, telephone, and in-person communications early on, the availability of increased data storage capacity, email communication, the Internet, and modernization of GPS and GIS technologies have allowed for large-scale coordination and logistical planning changes during the last 20 years. These new technologies have improved communication, transaction speeds, decision transparency, process documentation, and archiving. For example, survey materials are now electronically distributed and survey results are entered by the observer via the Internet (U.S. Fish and Wildlife Service 2017). The data are publicly accessible via the Internet, following a series of policy directives mandating broader public access to federal and federally-supported data and information (Executive Order 13642 2013 and OMB Memorandum M-13-13 2013). Moreover, GPS and GIS technologies have facilitated better route map management practices.
Technological advancements have facilitated better identification and understanding of potential issues with the SGS. For example, early comparisons of observer-submitted GPS coordinates (initiated in 2003) against official route maps indicated that many coordinates did not match the stop locations indicated on the map. Accurate spatial data are required to achieve various USFWS objectives: allowing researchers to show the relationship between woodcock abundance and landscape-level habitat variables, allowing the distribution of complete data coverages, and most importantly, enabling the USFWS to make more accurate and sound regulatory and management decisions in the future.
In light of changes in technology that allows for the assessment of route accuracy, and of indications from a recent study (Nelson and Andersen 2013) of unrecorded changes in route locations in Minnesota and Wisconsin, we evaluated the current accuracy in route-location records for the SGS. Our main objectives were to (1) assess all existing SGS route location spatial data and (2) quantify the common types of route map inconsistencies based on a review of SGS routes located in Minnesota and Wisconsin. Furthermore, we describe the challenges of maintaining SGS route consistency over time and provide recommendations on how to improve the SGS.

Study Area
All routes were located along lightly-traveled secondary roads in the center of randomly-chosen 10-minute degree blocks within each state and province in the central and northern portions of the woodcock's breeding range

Methods
To assess the accuracy of spatial data submitted by observers survey-wide, we first determined the status of spatial data for each route. We used route-location data from the route table within the SGS' relational database management system, which is administered by the national coordinator. The route table contained fields for each route (Table 1) indicating whether coordinates exist, the years GPS coordinates were submitted and for what stops, the verification status of the coordinates, and relevant verification user notes. From the table, we selected routes with 1 complete set of GPS coordinates. From within those routes, we created 3 groups: those where route coordi- nates were known to match the stop locations on the official route map, those where route coordinates were flagged with discrepancies or needing follow-up, and unreviewed stop coordinates. The verification process of each complete set (all 10 stops) of GPS coordinates was as follows: GPS coordinates were collected and submitted (U.S. Fish and Wildlife Service 2017) and entered online by the observer, state/provincial coordinator, or personnel within the Laurel USFWS office. These data went directly into the relational database management system that houses SGS spatial data. The national coordinator then compared these data to the official route map using a mapping program such as ArcMap (ESRI, ArcGIS Desktop, Redlands, CA, USA) or Google Earth Pro (Google, Inc., Mountain View, CA, USA). If documented as a correct match [count locations mirror that of the official survey route path, e.g., start and ending location match, count locations fall on the same roads and are equally distributed (0.6 km)] the route table was updated to reflect the accuracy of the GPS coordinates and a new map was created and distributed to the state/provincial coordinator and observer. The new map and GPS coordinates were stored in an electronic filing system and within route map binders at the USFWS' Laurel, Maryland, office.
We also examined the accuracy of spatial data from Minnesota (MN) and Wisconsin (WI) that had been previously reviewed and submitted to the national coordinator as part of a study assessing the relationship of woodcock counts with spatial variables measured along routes (Nelson and Andersen 2013). Nelson and Andersen (2013) chose a subset (n = 122 for MN and 65 for WI) of existing Minnesota (125) and Wisconsin (119) SGS routes in the Central Management Region and verified the loca-tion of these routes by asking the state coordinator to request that last known observers confirm route and stop locations derived from the official route maps or already existing verified GPS coordinates (supporting information Fig. S2). Based on specific observer feedback, Nelson and Andersen (2013) adjusted GPS coordinates and route maps and then provided these to the national coordinator for more detailed verification. Subsequently, we compared these maps (supporting information Fig. S2), revised based on observer feedback, to those maintained by state and national coordinators. We created a summary table for all routes within Minnesota and Wisconsin, regardless of whether they were included in the routes verified by Nelson and Andersen (2013), and recorded whether the verified stop coordinates not only matched those used in the study, but also matched the state and national coordinator official route maps. We noted whether the state coordinator and national coordinator route maps matched. Next, we incorporated any notes describing discrepancies associated with the GPS coordinates and/or route maps. We identified whether follow-up was needed for each route and who the last known observer was for the route in question. Next, we created a field that identified types of GPS coordinate discrepancies. The discrepancy types included: route run backwards or reverse order, route in a different location entirely, half the route on a different road, some or all stops have shifted, either or both the state/national coordinator map(s) not updated, the route did not yet officially exist (replacement routes), and clerical errors [GPS collection, transcription, or data entry errors or misinterpretation of route paths during the Nelson and Andersen (2013) map verification process]. We assumed that routes that fell into the first 4 category types above were incorrectly conducted by the observer at some point during their existence.

Results
We determined that stop-level GPS coordinates exist for 94% of all routes within the SGS, 66% of which have ≥1 complete set of verified GPS coordinates that match the official route map (Table 2). Our assessment indicated that 10% of the routes with GPS coordinates had discrepancies or required followup with state and provincial coordinators and/or observers. We did not analyze those 24% of routes (Table 2) with submitted GPS coordinates that had not been reviewed. We determined that 21% (Table  3)   Total in SGS 1,566 a In some cases a route has >1 set of submitted GPS coordinates. Incomplete sets were submitted often. Over the years, different observers were not aware that coordinates were already submitted and resubmitted them. Early on there was miscommunication and observers thought they were to submit coordinates each year they conducted the survey. ted by Nelson and Andersen (2013) did not match the official route map and/or the state coordinator map. This was double the rate (10%, Table 2) we found survey-wide for the routes that had been assessed to date. We determined that 80% of the GPS coordinates for Minnesota matched the official route map and/or the state coordinator map, whereas 20% were flagged as not matching (Table 3). Of that 20%, 42% were clerical errors on the part of the observer and/or Nelson and Andersen (2013) and were therefore adjusted (Table 4). For Wisconsin, we determined that 77% were verified as matching the official route map, whereas the remaining 23% were flagged because they did not match. Just over 13% of the 23% that were flagged sets of GPS coordinates were clerical errors on the part of the observer and/or Nelson and Andersen (2013) and were adjusted ( Table 4). The most common map discrepancies (Table 4) within the combined results for Minnesota and Wisconsin were shifted stop locations (26%) and clerical errors (31%). Fifteen percent of routes were in a different location entirely and 13% were run in reverse order. Both categories of routes with half the route on a different road and state and/or national coordinator maps not being updated were at 8% of discrepancies. Individual state results for Minnesota and Wisconsin are presented in Table 4.

Discussion
As indicated, 10-21% of the routes we assessed using stop coordinates submitted by observers did not match the official route maps. For Minnesota and Wisconsin, we were able to categorize these discrepancies (Table 4); however, we were not able to categorize these discrepancies for our survey-wide assessment (Table 2). Our perception is that the discrepancies for these routes would be similar to those we identified for Minnesota and Wisconsin (Table 4).
We are not certain why the discrepancies were double for the routes assessed in Minnesota and Wisconsin (21%) compared to nationally (10%). We speculate that possible reasons include: 1) 1 person coordinated these 2 states that included 243 routes, which was a significantly higher oversight percentage than any other state or provincial coordinator, 2) most states and provinces were coordinated by a state or provincial employee who often had closer oversight for observers (mostly state and provincial employees, themselves) than a federal coordinator, and 3) lack of coordination on route issues between individual observers and the state coordinator.
Maintaining route consistency for an established, longterm, national roadside survey is a large effort that requires consistent communication and cooperation between and among individuals at all participation levels. Consequences occur when there are communication breakdowns, and these consequences compromise the quality of the survey and its data. To improve the integrity of the survey and its data, not only within Minnesota and Wisconsin, but for all states and provinces within the SGS, we recommend improving communication and coordination efforts. The development of robust SOPs, documented route changes, improved training methods in the form of workshops and webinars, and utilization of existing resources and available technology are all methods that will improve communication and coordination, and in turn, increase the accuracy of SGS data.   (19) and Wisconsin (66) routes because they did not get a response back from the observer or the routes did not fit into their study design: route did not exist yet, was discontinued, was not surveyed that year/recently, numbers heard were zero, or they were in a constant zero status (Seamans and Rau 2017) at some point during the years of interest.

RoBuST SoPS And doCumenTATion
In the case of the SGS, verifying the accuracy of spatial data for each survey route is time-consuming, and the effort of the national coordinator to verify spatial data is limited by the level of existing resources. Enlisting the assistance of observers who submit data is the first step in ensuring the data are accurate. Requiring observers to map their GPS coordinates before submitting them to the national and state/provincial coordinator will prevent GPS collection, transcription, or data entry errors, identified as one of the most common map discrepancies (Table  4). Passing all submitted GPS coordinates through an approval process at the state or provincial level before submission to the national coordinator would prevent other route map discrepancies (Table 4). This process can prevent mismatched route maps between the national coordinator and the state/provincial coordinator (Table 4). Ideally, route maps should be made by 1 agency, preferably within the USFWS' Laurel, Maryland office from which the SGS is coordinated. This would alleviate route map inconsistency among states and provinces as demonstrated in supporting information Fig. S1 and Table 4. If this is not feasible, however, a SOP should be developed that specifies the requirements for route maps. For example, all maps should indicate the state/province; route number; county code; date of creation; date of collection; start, finish and stop numbers along the route; correct GPS coordinates for each stop; street names of starting intersection (if one exists); creator of route map; and, if relevant, the number of the route it replaced and year of replacement.
When route changes occur, there needs to be an update of route information, including route relocation metadata that includes years of discontinuation, why it was discontinued, what route number replaced it, and who developed the new route. Although documentation exists since 2007 for almost all authorized route changes, it is not organized to be easily attainable or distributed. This lack of adequate documentation leads to route map or route number confusion by the observer, which can lead to observers conducting discontinued route paths rather than the replacement route. Although this route discrepancy was not specifically addressed in our results, because it can overlap multiple discrepancy categories, it likely occurs and can be prevented with additional information. In the case of Seamans and Rau (this volume), to estimate a potential annual indices bias of discontinuing routes and replacing them with new routes, a count comparison of woodcock at discontinued routes with counts from their replacement routes was needed. Historically, route relocation metadata were not documented in a table form and had to be created so they could be used to determine which routes were discontinued and what their replacement route numbers were. Such a table might also have been helpful to Nelson and Andersen (2013) as a source of discontinued routes when they were defining their study design.

TRAining And woRKShoPS
Many of the route map discrepancies identified in Table  4 have occurred because observers were either unaware of protocols for route changes or were unaware of the ramifications of unauthorized route changes. One way to mitigate miscommunication among the many people who coordinate and conduct the survey is to develop training presentations or online videos that describe the survey and its importance to woodcock management. The USFWS developed a required training presentation for all observers to view. The presentation details the responsibilities of the observer and the SOPs they must follow. It outlines why observers need to follow specific guidelines and highlights the consequences if they do not. Videos would be beneficial because observers can be shown conducting the survey, demonstrating common scenarios that often lead to stop location discrepancies demonstrated in Table  4. They can also offer up consistent solutions that observers might encounter during the survey, such as how to proceed when a stop is no longer safe or what to do in the event that a road is blocked.
Route inconsistency is often a product of high observer or office turnover, and of new observers being unfamiliar with the route or stop locations. For example, stops along their route may not been georeferenced, which is the case for 6.2% of the 1,566 routes in the survey, or the GPS coordinates associated with that route may not have been reviewed or verified, which is the case for 23.6% of the 1,469 routes with GPS coordinates (Table 2). Although the SGS protocol recommends that during a year of observer transition, the old and new observer run the route together, this is not always possible. To minimize the possibility of errors, we highly encourage coordinators to ask observers if they are willing to survey routes in future years, and check on observer status periodically to ensure they have time to find replacements or to create a transition team when needed. Ensuring coordinators are up to date on the participation status of observers also allows time for the newly selected observer to scout the route prior to a survey, review the official route map, and ask questions if they are not clear on SGS guidelines.
Conducting periodic training sessions in the form of webinars and workshops for state and provincial coordinators and observers could prove beneficial. These can be used as a forum to highlight specific SGS logistical concerns and would provide an opportunity for group discussion of resolutions. Training was recommended to Breeding Bird Survey (BBS) observers by a peer-reviewed panel after a programmatic review of the North American BBS, another long-standing roadside survey (O'Connor et al. 2000). That review highlighted how individual coordinators had very different approaches to meeting their responsibilities within the BBS and thus could create openings for complementing their efforts with respect to observer recruitment and retention. Improving communication among individuals will yield insight into new ways to stimulate participation and commitment.

uTilizing exiSTing ReSouRCeS oR AvAilABle TeChnology
At the national coordinator level, resources and staff will be limited into the future. A partial solution to the backlog of GPS coordinates that need to be verified (Table 2) could be working with a volunteer or intern. Enlisting the help of the state or provincial coordinators to help verify or remedy specific flagged routes would also help speed up the verification process. Currently, each state and provincial coordinator receives an annual report indicating the GPS coordinate status for each route. The report indicates which routes have verified GPS coordinates and which have flagged GPS data and for what reason. This report should be distributed to observers, who can make corrections.
Another potential tool to increase the proportion of routes and stops with accurate spatial data would be development of an online system that allows all official route maps to be obtained through the Internet. Such a system could include a tool that allows not only for the entry of spatial data, but for verification of such data against the assigned route map already in the online system. Developing such an online system that includes a map repository and access to GPS coordinates could make the same current route maps available to all parties. Such a system would preclude having to verify route information before analysis, as was the case for Nelson and Andersen (2013). Developing an application that allows for the download of route maps to a mobile or GPS device would also be helpful in deciphering route path direction, a common route map discrepancy we detected. A system that facilitates map access for portable devices would also allow observers to instantly see if GPS coordinates are correct, and would assist observers who are submitting GPS coordinates for replacement routes.
Until the advent of an online system for route map verification, conducting periodic verification of route and stop locations would help minimize inaccurate location data. Such verification could start with comparing national coordinator route maps against state and provincial route maps followed by asking observers to verify the accuracy of maps.
Currently, survey materials are distributed (U.S. Fish and Wildlife Service 2017) through email from the national coordinator directly to state and provincial coordinators. The coordinators then distribute these materi-als directly to observers (via email or mail) or through another regional level within each state or province before reaching each observer. With the numerous steps and large volume of individuals involved in survey distribution there is a chance that not all materials will make their way to all observers. Placing all survey materials in 1 place on the Internet for direct access and download/upload would save time and effort, and would create a more direct line of communication. It would also discourage route map mismatch discrepancies. This could be implemented rather quickly if it is done at the same website where observers enter their survey data.
Although not directly part of this assessment, another issue we identified is the annual survey of routes. During the Nelson and Andersen (2013) study (Table 3), a large number of Wisconsin routes were not adequate for their study design, primarily because the routes were not surveyed during the study years (M. R. Nelson, Fish-Lectronics, personal communication). In 2007, the USFWS distributed a survey participation assessment report (unpublished) that identified areas where many routes were not surveyed. Incomplete routes often resulted from states and provinces lacking the resources to devote to the survey. Within Ontario, for example, 75% of the routes were not surveyed at the time the participation report went out. Subsequently, Bird Studies Canada, a volunteer citizen-science charity-based organization, was engaged to help administer the SGS and solicit volunteers for routes in areas where the Ministry of Natural Resources and Forestry observers lacked coverage. The number of conducted routes increased as much as 45% over the next 5 years. We recommend engaging similar citizen-science groups that can assist with the completion of survey routes in low participation areas.

Management Implications
While numerous challenges exist in the coordination of any long-term, large-scale survey, route inconsistency is a particular problem for the SGS. Finalizing the verification of spatial data for all routes is crucial especially because spatially explicit data are essential for assessing the relationship of counts to habitat variables at multiple scales. We will continue our efforts to address discrepancies in the spatial data that have been submitted to date and continue to obtain spatial data for the remaining SGS routes with no stop-level coordinates. Implementing the recommended solutions will maximize route consistency through time and improve the overall quality of the survey data, thus enabling the USFWS to make more accurate and sound regulatory and management decisions.