Evaluation of Statistics Canada’s 2021 Census of Agriculture

Evaluation Report

February 2024

How the report is structured

• The report in short
• Acronyms and abbreviations
• What is covered
• What we learned
  • 1. Performance—achievement of expected outcomes
  • 2. Performance—planning process
  • 3. Performance—communication strategy and activities
• How to improve the program
• Management response and action plan
• Appendix A – Interview quantification scale

The report in short

Statistics Canada has a mandate and legal obligation under the Statistics Act to conduct the Census of Agriculture (CEAG) every five years. The CEAG provides a comprehensive profile of the physical, economic, social and environmental aspects of Canada's agriculture industry. It is the only data source that consistently provides high-quality detailed statistical information on agriculture for small geographic areas and collects a wide range of data at the national, provincial and subprovincial levels.

The CEAG provides both a snapshot in time of the agriculture industry and changes over time that are essential to inform public and private decision making. Its expected outcomes are to provide high-quality data and information that is accessible, available and relevant to all data users.^{Footnote 1} In addition to achieving the CEAG outcomes, the 2021 CEAG was expected to

• decrease response burden on farm operators
• increase integration with Statistics Canada's centralized infrastructure
• increase harmonization between the methods, concepts and processes of the CEAG and other economic statistics programs
• publish more data without compromising the confidentiality of respondents
• align the business model with Statistics Canada's modernization objectives.^{Footnote 2}

The evaluation was conducted in accordance with the Treasury Board Policy on Results and Statistics Canada's Risk-Based Audit and Evaluation Plan (2023/2024 to 2027/2028). The objective of the evaluation is to provide credible and neutral information on the ongoing performance of the CEAG and to identify potential areas to consider for the CEAG's next cycle.

The evaluation methodology consisted of a document review and interviews. Interviews were conducted with Statistics Canada staff, as well as with data users external to Statistics Canada. The findings outlined in this report are based on the triangulation of these data collection methods.

Key findings and recommendations

The CEAG program achieved its expected outcomes of providing high-quality data and information that is accessible, available and relevant to all data users with the timely delivery of the 2021 CEAG.

Compared with 2016, the 2021 CEAG both sustained and improved on various factors that contribute to producing high-quality data and to their relevance to users, their availability and their accessibility. The 2021 CEAG made progress on its objectives to improve operational efficiencies through integration with Statistics Canada infrastructure, increase harmonization with other economic statistics programs and reduce response burden. As with transformational changes, full potential benefits will be realized moving forward as the processes and activities become fully integrated and more familiar to staff.

In general, the main overarching planning processes for the 2021 CEAG were similar to the 2016 cycle, with the addition of several significant methodological and operational changes: the change in the farm definition, the migration to the Integrated Business Statistics Program, the continued development of the electronic questionnaire and adaptations made because of the pandemic. The risk management process contributed to achieving expected results, as well as mitigating the risks that are specific to the changes made in the 2021 cycle. The significant number of changes for the 2021 CEAG made planning more challenging because of some unpredictable implications from the changes, and staff turnover added additional planning constraints. These large-scale changes implemented for the 2021 CEAG have also fundamentally altered many processes, establishing a new baseline moving forward.

Overall, the 2021 CEAG communication strategy and activities were effective in maintaining, and in some cases improving, awareness of CEAG products. New and improved tools increased awareness and facilitated users' understanding of how to interpret CEAG data. Some areas noted for consideration for subsequent cycles of the CEAG are continuing to amplify the CEAG through various communication and engagement channels; empowering users to access, use and interpret the data; and providing clearer information and guidance on changes.

In light of these findings, the following recommendations are proposed:

Recommendation 1

The Assistant Chief Statistician (ACS), Economic Statistics (Field 5), should

1. ensure that the process in place to regularly review resources includes revisiting the balance between upcoming and planned deliverables (including those for communication and engagement) and the corresponding challenges versus available time and resources
2. in addition to the regular review process, given that the 2026 CEAG is not expected to include any new large-scale changes, verify that the planned levels and allocation of resources are correctly aligned with the new baseline resulting from the changes made for the 2021 CEAG; this will help ensure that future core activities are resourced appropriately.

Recommendation 2

The ACS, Economic Statistics (Field 5), should ensure that effective training, retention and succession plans are developed, reviewed regularly and aligned with future needs of the CEAG program.

Recommendation 3

The ACS, Economic Statistics (Field 5), should ensure that the CEAG has a communication and engagement plan in place that includes

1. varied communication approaches to broaden reach and clearly communicate the CEAG's intentions and plans, including the corresponding benefits, to increase stakeholders' understanding and acceptance
2. adequate approaches to improve data users' awareness of and engagement with CEAG data, to further support CEAG tool usage and data interpretation, including working with users with less statistical capacity.

Acronyms and abbreviations

ACS

Assistant Chief Statistician

ASP

Agriculture Statistics Program

ATDP

Agricultural Taxation Data Program

CAMT

Census of Agriculture Management Team

CASC

Census of Agriculture Steering Committee

CEAG

Census of Agriculture

CePop

Census of Population

DPMO

Departmental Project Management Office

EQ

Electronic questionnaire

FPT

Federal-Provincial-Territorial

IBSP

Integrated Business Statistics Program

NFD

New farm definition

OMG

Operations Management Group

PMT

Project Management Team

SMC

Strategic Management Committee

What is covered

The evaluation was conducted in accordance with the Treasury Board Policy on Results and Statistics Canada's Risk-Based Audit and Evaluation Plan (2023/2024 to 2027/2028). The objective of the evaluation is to provide credible and neutral information on the ongoing performance of the Census of Agriculture (CEAG) and to identify potential areas to consider for the CEAG's next cycle.

The Census of Agriculture

Mandate, outcomes and objectives

Statistics Canada has a mandate and legal obligation under the Statistics Act to conduct the CEAG every five years. The CEAG provides a comprehensive profile of the physical, economic, social and environmental aspects of Canada's agriculture industry. It is the only data source that consistently provides high-quality detailed statistical information on agriculture for small geographic areas and collects a wide range of data at the national, provincial and subprovincial levels. The types of data collected include the number of farms and farm operators, farm area and size, land management practices, business operating arrangements, farm operating revenues and expenses, and farm capital. The CEAG provides both a snapshot in time of the agriculture industry and changes over time that are essential to inform public and private decision making.

The expected outcomes for the CEAG are to provide high-quality data and information that is accessible, available and relevant to all data users.^{Footnote 3} To achieve its outcomes, the CEAG is expected to

• provide statistical information and analysis about Canada's economic and social structure to develop, evaluate and improve public policies and private decision making
• promote sound statistical standards and practices to achieve greater efficiency in data collection
• provide critical information to manage federal and provincial governments' expenditures in support of the agriculture sector
• benchmark agriculture information to realign annual and subannual survey estimates and economic data to ensure accuracy and coherence
• provide agriculture information for small geographic areas based on complete enumeration to inform program and policy monitoring
• provide an accurate frame of all farms and farm types to inform the agriculture survey program
• provide measurement of rare or emerging commodities to inform disease control or trade issues.

In addition to achieving the CEAG outcomes, the 2021 CEAG was expected to

• decrease response burden on farm operators
• increase integration with Statistics Canada's centralized infrastructure
• increase harmonization between the methods, concepts and processes of the CEAG and other economic statistics programs
• publish more data without compromising the confidentiality of respondents
• align the business model with Statistics Canada's modernization objectives.^{Footnote 4}

Census cycle

The CEAG is conducted in May to align with the Census of Population (CePop) and take advantage of cost savings achieved by leveraging the CePop's resources and infrastructure. Every CEAG census cycle has a six-year timespan, during which Statistics Canada conducts several activities. Figure 1 below highlights the six broad components that make up the CEAG's six-year cycle. While the activities are presented in a linear progression, many activities across projects happen simultaneously.

Description - Figure 1: The six components and associated key activities of the Census of Agriculture cycle

Figure 1 depicts the six broad components that make up the CEAG's six-year cycle.

1. Content determination: 
   • Consult with user groups across the country to inform Census of Agriculture (CEAG) questions (e.g., identify data gaps and needs of data users).
   • Test content changes, including the proposed questions, electronic questionnaire, edit and imputation processes, and validation tools.
   • Seek Cabinet approval and publish CEAG content in The Gazette.
2. Communications:
   • Develop and launch a campaign to raise awareness of the CEAG, promote self-response through online collection and increase self-response rates.
   • Promote messaging through various communication channels, including outreach, public and media relations, social media, respondent relations, and census website management.
3. Collection:
   • Develop the census frame using Statistics Canada's Business Register and tax remittances.
   • Collect data online or (on demand) on paper (questionnaire forms) from May 3 to August 29, 2021.
   • Distribute email, letter and voice reminders.
   • Follow up with non-respondents to complete the questionnaire.
4. Processing:
   • Conduct initial processing: geo edit to assign province or territory, census agricultural region, division and subdivision; code open-ended responses; remove duplicate questionnaires; and link farm operators with people in the Census of Population database.
   • Import data into the Integrated Business Statistics Program and integrate them with other data sources, address data errors through edit and imputation, and generate estimates through summation.
5. Data quality evaluation:
   • Validate data by finding and correcting significant errors at the provincial and subprovincial levels, then at the farm level, and finally conduct a macro-level review to assess final estimates.
   • Develop and present certification reports summarizing the data validation process and recommendations for publication.
6. Dissemination:
   • Develop and release informational products, including farm and farm operator data, provincial and territorial profile articles, analytical articles, agriculture-population linkage data, the Guide to the Census of Agriculture and reference and thematic maps, a mapping tool, and community profiles.
   • Generate and distribute custom data products requested by data users.

Governance structure

The CEAG is overseen by the Strategic Management Committee (SMC), a senior governance body chaired by the Chief Statistician and composed of assistant chief statisticians (ACSs).

The Census of Agriculture Steering Committee (CASC) reports to the SMC and directs the Census of Agriculture Management Team (CAMT). The CASC is chaired by the Director General, Agriculture, Energy and Environment Statistics Branch (the business sponsor), and consists of the CEAG manager and directors from various Statistics Canada divisions and stakeholders from Agriculture and Agri-Food Canada. Beyond providing direction to the CAMT, the CASC is responsible for monitoring project progress; approving recommendations for changes to scope, schedule and cost; and providing advice and assisting with the resolution of issues between stakeholder groups.

The CAMT is responsible for the ongoing decision making and management of the CEAG, including risk management and mitigation planning. The CAMT is composed of the Census Manager and subproject managers (e.g., content determination, collection, processing, validation, certification and dissemination). The CAMT meets weekly and as necessary, and subproject teams meet biweekly and report progress and issues to the CAMT.

External consulting bodies engaged in the CEAG include the Advisory Committee on Agriculture Statistics and the Federal-Provincial-Territorial (FPT) Committee on Agriculture Statistics. These bodies provide feedback on emerging issues and informational needs within the agricultural sector.

Internal management committees also help direct the CEAG. The Economic Statistics Field Planning Board reviews new project proposals, is responsible for making final decisions for reviews of all projects in the field, and monitors project execution and the allocation of information technology resources. Finally, initiatives related to the Integrated Business Statistics Program (IBSP) fall under the IBSP Project Management Team (PMT).

The evaluation

The scope of the evaluation covered ongoing program performance, including changes made between the 2016 and 2021 census cycles, as well as performance from the lens of planning and communications. The scope was established in collaboration with the office of primary interest. The evaluation was conducted from July to October 2023.

The three evaluation questions that were identified are the following:

1. To what extent has the 2021 CEAG achieved its expected outcomes?^{Footnote 5}
2. To what extent did the 2021 CEAG's planning process effectively support the achievement of expected outcomes?
3. To what extent have the 2021 CEAG's communication strategy and activities effectively supported the achievement of expected outcomes?

The data collection methods outlined in Figure 2 were used.The findings outlined in this report are based on the triangulation of these data collection methods.

Description - Figure 2. Data collection methods

Figure 2 depicts the three data collection methods used for the evaluation: external interviews, internal interviews, and document review.

The external interviews included semi-structured interviews data users from provincial government organizations, academia and agriculture organizations. There were 20 interviews conducted with 26 people.

The internal interviews included semi-structured interviews with Census of Agriculture representatives, as well as partners within Statistics Canada. There were 8 interviews conducted with 8 people.

The document review included a review of Statistics Canada's files, documents, and web trends information.

Two main limitations were identified, and mitigation strategies were employed, as outlined in Table 1.

Table 1. Limitations and mitigation strategies

Limitations	Mitigation strategies
The perspectives gathered through external interviews may not be fully representative.	External interviewees were selected using specific criteria to maximize strategic reach for the interviews. Multiple recruitment strategies were used. Evaluators were able to find consistent overall patterns.
There were a few internal interviewees who were unavailable to participate, and therefore, their perspectives are not represented in the evaluation.	To address this gap, documents—particularly the task evaluation reports—were used to provide additional evidence, and alternate interviews were conducted.

What we learned

1. Performance—achievement of expected outcomes

Summary

The CEAG program achieved its expected outcomes by providing high-quality data and information that is accessible, available and relevant to all data users with the timely delivery of the 2021 CEAG. Compared with 2016, the 2021 CEAG both sustained and improved on various factors that contribute to the expected outcomes. It made progress on its objectives to improve operational efficiencies through integrating with Statistics Canada infrastructure, increasing harmonization with other economic statistics programs and reducing response burden. As with transformational changes, full potential benefits will be realized moving forward as the processes and activities become fully integrated and more familiar to staff.

The 2021 CEAG was expected to provide high-quality statistical information that is relevant to its users, available and accessible. In addition, the 2021 CEAG intended to reduce respondent burden, improve operational efficiencies^{Footnote 6} through integration with Statistics Canada infrastructure and increase harmonization with other economic statistics programs.

Compared with 2016, the 2021 CEAG both sustained and improved on various factors that contribute to producing high-quality data and to their relevance to users, their availability and their accessibility.

Improvements to data quality resulted from the promotion and uptake of the online electronic questionnaire (EQ), achieving 82% uptake and exceeding the target by 12%. The uptake of the EQ contributes to data quality by preventing certain respondent errors. For example, when unlikely or invalid data were entered, the EQ initiated a prompt for the respondent to check their response, and the EQ automatically filled in certain fields based on information from the Business Register. Other factors contributing to data quality include the response rate and coverage errors. While the 2021 response rate was lower than in 2016, and the 2021 farm count undercoverage rate was higher than in 2016, neither of these factors is directly comparable between the two CEAG cycles because of new methodologies used in 2021.^{Footnote 7}

Overall, the 2021 CEAG data coverage and relevance were similar to 2016, with some improvements resulting from the content consultations with census stakeholders (federal departments, provincial ministries, agriculture associations, etc.). The addition and refinement of questions about sustainability practices and technology adoption in the agriculture sector improved the breadth and depth of data coverage, as well as relevance to data users.

The timeliness of releases for some of the 2021 CEAG products was improved, compared with the 2016 CEAG cycle. New products, including reference maps, a mapping tool and the CEAG Guide, were released one to three months ahead of the farm and farm operator data. The timing of the release for the first key CEAG data output, the farm and farm operator data, was similar to that of the 2016 cycle—one year following Census Day, which is the day that the census is rolled out. The following CEAG outputs were also released earlier, compared with the 2016 cycle:

• the agriculture-population dataset that links CEAG and CePop data was released three months earlier
• the reference maps, which provide the geographical boundaries, codes and names for all geographic areas appearing in the data tables, were released approximately three months earlier
• custom data orders requested by end users were reported to be released faster in 2021.

While there were a few end users who reported that timelier CEAG data could better meet their needs, or questioned why the transition to an EQ did not result in faster data releases, the general consensus was that there is always a preference for high-quality data over earlier data, and the timelines for CEAG data were appropriate.

The availability of data was improved by publishing (for the first time) quality indicators in the farm and farm operator data to help data users determine to what extent they may want to use certain data points to inform decisions, policies or programs. Data availability was further improved through the introduction of a new disclosure methodology^{Footnote 8} that decreased data suppression—a noted improvement for a few key external interviewees. Improvements to data access included making microdata available in research data centres for researchers with research proposals to access directly. Additional accessibility improvements are discussed in the "Performance—communication strategy and activities" section.

The maintenance of and improvement to elements of data quality were facilitated by the promotion of an EQ, the content consultation process and the experience level of data users.

As already stated, data quality improvements were facilitated by the promotion and uptake of the EQ. Data coverage and relevance improved too, because of the content consultations that informed questions about sustainability practices and technology adoption. In addition, consultations following the 2016 CEAG cycle with key stakeholders (e.g., federal and provincial or territorial governments, Agriculture and Agri-Food Canada) facilitated improvements in data access and use through the addition of new data visualization and mapping tools. Custom datasets were reported as accessible and timely for long-time data users, given that the requested data were often similar from cycle to cycle and the process was well understood. For less experienced CEAG users (and some smaller producer organizations), the process (e.g., where to direct a request, how to frame the data request) and costs associated with requesting custom data were viewed as barriers.

Operational and methodological changes tempered some of the improvements to data quality and coverage, while other improvements to data quality and availability were facilitated by operational changes.

The methodological change to calculating the CEAG coverage errors generated higher rates of undercoverage; however, the 2021 estimates are considered to be more accurate as a result of the improved methodology. Although planned additional improvements to include detailed revenue data from the Agriculture Taxation Data Program (ATDP) for publication did not go forward because of concerns related to accuracy, the ATDP data replacement improved overall data coverage and quality because it avoided missing values from refusal or false information. Furthermore, the migration to the IBSP enabled the publication of data quality indicators and, as a result, improved data availability.

The 2021 CEAG made progress on its other objectives to improve operational efficiencies through integration with Statistics Canada infrastructure, to increase harmonization with other economic statistics programs and to reduce response burden. Full potential benefits will be realized moving forward as the processes and activities become fully integrated and more familiar to staff.

Operational efficiency gains were realized through the migration of the 2021 CEAG to the IBSP. The migration to the IBSP supports operational efficiencies through data integration from other programs; earlier data validation because of rolling estimates; and the use of an existing corporate tool, expertise and experience, as opposed to a stand-alone processing system. From the perspective of internal interviewees, the new processes, approaches and sustained effort required to address the challenges^{Footnote 9} related to the IBSP migration limited the gains for the 2021 CEAG cycle, but the lessons learned from the migration are expected to yield more efficiencies for subsequent CEAG cycles.

The introduction of a new farm definition (NFD) for the 2021 CEAG increased harmonization with other economic statistics programs.^{Footnote 10} This NFD facilitated the flow of data from other economic programs, such as the ATDP, into the CEAG and supports the comparison and interpretation of data across all agriculture statistics programs that share the same definition. Furthermore, the NFD supports harmonization with tax data and the Statistics Canada Business Register, used to develop the census frame.

The 2021 CEAG decreased response burden through the use of administrative data replacement, filter questions, an optimized online EQ and the use of tax data to inform the sample frame. Data replacement decreased response burden by using existing data to replace questions about revenues and expenses, sex, age, and operating arrangement. Filter questions reduced response burden, since only questions that were relevant to each respondent's operations were asked. Finally, the new census frame decreased response burden for out-of-scope operations (e.g., hobby farms) that no longer receive the CEAG and for CePop respondents who no longer had to answer a question about agriculture revenue. While data replacement theoretically reduced the time to complete the 2021 CEAG by 10% (or 3.3 minutes), the actual time to complete the CEAG decreased by 30 seconds because of additional qualifying questions designed to ensure that respondents were in scope.

The greater integration with Statistics Canada infrastructure and the harmonization with economic statistics programs were both facilitated and hindered to varying degrees by similar factors: the experience level of team members, project management and the COVID-19 pandemic.

Internal interviewees and the documentation review detailed facilitators and hindrances to the tasks involved with, or resulting from, IBSP integration and harmonization with economic statistics programs, such as the following:

• Some activities involved with the migration to the IBSP (e.g., taxes and modelled births added into the population and imputed into the IBSP) were facilitated by strong collaboration and consultation, while other activities could have benefited from more consultation and collaboration with other Agriculture Statistics Program (ASP) teams (e.g., validation of revenues and expenses integrated from the ATDP).
• The presence of some experienced CEAG team members and ASP staff facilitated the necessary process adaptations or workarounds required to accommodate the migration to the IBSP; however, the data validation activities caused several challenges for the validation team, which did not have prior experience with the new iterative process and the interdependencies between detailed, subtotal and total values in the IBSP.
• The larger staff recruitment pool (pan-Canadian) resulting from the remote work policies related to COVID-19 facilitated some production-related activities; however, remote training challenges early on in the pandemic hindered training.
• Some activities benefited from well-defined project schedules, roles and responsibilities (e.g., the NFD to harmonize with economic statistics programs and integrate ATDP data within the IBSP), while for others, there was a feeling that more planning and management would have been beneficial (e.g., last-minute organizational plan for validation of revenue and expense data).

The commitment of staff to the CEAG helped teams address the various challenges associated with the 2021 CEAG changes, but tight timelines did not permit sufficient opportunity to research, develop, test, analyze and revise some activities as thoroughly as staff would have liked (e.g., backcasting, data validation, NFD and sampling frame), leading to unexpected challenges that required additional work, including overtime.

The 2021 CEAG methodological and operational changes impacted the CEAG work environment, and changes to decrease response burden affected data processing.

The number and complexity of operational and methodological changes implemented in 2021 contributed to a heightened level of stress for many team members.

The transition to the new environment required several workarounds that were largely understood and developed by staff who had extensive experience with the CEAG program—many of whom have moved on, or (as of the fall of 2023) are expected to move on, from the program. This gap in institutional knowledge creates challenges for the 2026 Census cycle, which will have fewer team members with experience in the full six-year census cycle.

In addition, the EQ filter questions decrease response burden by asking a series of yes or no questions to determine what questionnaire sections apply to each respondent. However, the number of unexpected answers to these questions was higher than anticipated, resulting in more corrections and higher imputation rates. For example, it was found that a high number of respondents indicated that they had no machinery. This then required the data validation team to identify records that likely did have machinery and manually fix the answer before sending the records to imputation.

The replacement of some CEAG respondent data with those from other administrative sources (part of AgZero^{Footnote 11}) is advancing one of the five pillars of the agency's modernization agenda. Some external interviewees relayed their concern that transitioning to administrative or survey data may limit data at smaller geographic levels, where administrative data may not be available.

2. Performance—planning process

Summary

In general, the main overarching planning processes for the 2021 CEAG were similar to the 2016 cycle, with the addition of several significant methodological and operational changes: the change in the farm definition, the migration to the IBSP, the continued development of the EQ and adaptations made because of the pandemic. The risk management process contributed to achieving expected results, as well as mitigating the risks that are specific to the changes made in the 2021 cycle. The significant number of changes for the 2021 CEAG made planning more challenging, and staff turnover added additional constraints.

The main steps of CEAG planning were the same as in the previous cycle. The addition of several significant methodological and operational changes affected planning: the change in the farm definition, the migration to the IBSP, the continued development of the EQ and adaptations made because of the pandemic.

The overall governance for the CEAG planning process for the 2021 cycle was similar to the 2016 cycle. However, the Operations Management Group (OMG) was introduced in March 2021 to manage the CEAG production. The OMG core members^{Footnote 12} met daily from April 29, 2021, to March 4, 2022, to identify and address CEAG production-related issues. As well, the inclusion of the IBSP PMT provided oversight for all programs migrating to the IBSP framework. Furthermore, the following revisions to the organizational structure occurred in the 2021 cycle:

• the CEAG dissemination unit expanded to do more divisional work
• CEAG processing responsibilities shifted to the Enterprise Statistics Division, which is responsible for the IBSP
• client services (i.e., custom data) transitioned to Statistics Canada's regional data service centres.

The main overarching planning processes for the 2021 CEAG were similar to the 2016 cycle. The content determination project included its usual national consultation process to inform the instrument design with data users in 2017; questions were tested, and the CEAG team continued to explore steps that could be partially or entirely replaced by administrative data. The communication project established a plan to increase awareness and promote participation in the CEAG, while leveraging the communication efforts of the CePop. The collection, data quality and dissemination projects all developed strategies and plans to ensure that their activities were successfully executed. Planning oversight by the CEAG management teams, including the CAMT and unit heads, continued through regular meetings over the course of the CEAG cycle.

In addition, the 2021 planning process had to accommodate several large-scale changes, some of which were related to the agency's modernization and innovation agenda. Specifically, the 2021 CEAG had to plan for the following changes:

• implementing the NFD
• increasing data integration
• executing a new data processing environment
• pursuing further development of the EQ to reduce response burden
• realizing a new disclosure avoidance method.

Planning started prior to the Treasury Board submission for the 2021 CEAG cycle, specifically for activities involving the NFD and migration to the IBSP, and new methodologies were developed for CEAG data validation to accommodate the IBSP circular processing model. However, a few internal interviewees reported that, as a result of the extent and complexity of the changes, they could have benefited from more time to plan and implement them.

The 2021 CEAG risk management process contributed to the achievement of expected results by identifying and planning for risks that are standard to most CEAG cycles, as well as risks specific to the changes planned for the 2021 cycle.

Throughout the CEAG cycle, a risk register was managed by the CAMT, and risks were reported on at the monthly CASC meeting. Risks unique to the 2021 CEAG were primarily related to harmonization with other economic statistics programs, namely the new census frame; the NFD; and migration into the IBSP. These risks, along with their predicted impact and associated mitigation plans, are further described below.

• New census frame: Risks identified with the new definition included potential impacts on data quality. The exclusion of in-scope farms from the census frame was a potential data-quality risk resulting from the NFD and dependency on the Business Register to inform the frame. To mitigate this risk, the new frame was tested against the 2016 Census frame and the Agriculture Frame Update Survey to ensure alignment, and modelled births were included in the CEAG to account for new farms since the 2020 tax data.
• NFD: The effect of the NFD on the comparability of 2021 CEAG data with 2016 CEAG data was a data relevance risk from end users' perspective. To mitigate this risk, a backcasting exercise was undertaken to better understand how to interpret the 2021 CEAG data, compared with 2016.
• Migration to IBSP: One of the key risks reported by a few internal interviewees regarding the IBSP migration was related to the IBSP's ability to process the volume of data involved in the census, possibly disrupting the timelines. To mitigate this risk, the CEAG was tested repeatedly in the IBSP environment prior to production, and CEAG teams consulted with other teams that had undergone the transition to the IBSP, as well as the IBSP managing team.

Other risks classified as being low or moderate and, for the most part, deemed manageable included low response rates, challenges in securing enough skilled human resources at the right time and natural disasters.

In addition to the planned risks, the 2021 CEAG also needed to account for unanticipated risks, namely those pertaining to the COVID-19 pandemic. While the program has a business continuity plan, it did not account for the scale and ramifications of the pandemic. The planning for a variety of 2021 CEAG activities had to pivot because of COVID-19. For example,

• the communication and engagement plan transitioned from including in-person engagement (e.g., attendance at agricultural shows) to being completely online
• the data replacement plan was revised to ensure that industry data impacted by the pandemic were excluded from the replacement strategy
• the training plan was revised to be online and accommodate a pan-Canadian workforce working from home (as opposed to onsite in the National Capital Region).

Facilitators in CEAG planning included the matrix management approach and regular planning meetings. The significant changes for the 2021 CEAG made planning more challenging, given the complexity and magnitude of the changes. Staff turnover was an additional challenge.

The CEAG planning process used a matrix management approach to help ensure the interdependencies across divisions and activities outside the Census Manager's reporting structure were kept on track. A few internal interviewees reported that the regular planning meetings between the unit heads for collection, processing and dissemination, as well as cross-unit planning meetings between the CEAG and CePop, also facilitated the planning process. As already stated, the planning around the IBSP migration was facilitated by consultations with other teams that had undergone the transition. Because of the unique and complex nature of the CEAG, there were unexpected challenges (e.g., dealing with the infrastructure of the processing environment, CEAG descriptive mnemonics versus IBSP naming conventions) that the CEAG team had to address that were not experienced by other teams.

The substantial changes introduced in the 2021 CEAG cycle (e.g., IBSP migration and NFD) were reported to be the most significant planning challenge. A few internal interviewees reported that they felt that the complexity of the changes was not adequately planned for, despite factoring in the potential challenges in the planning process. Other challenges included staff turnover, especially at the critical levels (e.g., key management personnel and subject-matter experts), that left gaps in knowledge and hindered planning.

For the amount and level of change implemented for the 2021 CEAG, resources were viewed as being very tight for the core CEAG team.

Significant effort went into planning for the 2021 cycle. In addition, efforts to identify lessons learned from the various tasks for the 2021 CEAG cycle provided detailed feedback to inform planning for future cycles. Some internal interviewees reported that there was insufficient time to fully plan for the unprecedented scale of changes. It was suggested by some interviewees that incorporating the significant changes more gradually would have alleviated some pressure; however, it was noted during a subsequent discussion that gradual implementation across more than one cycle was not possible for the 2021 CEAG because of the interrelated nature of the changes.

The 2021 CEAG also experienced human resource challenges throughout the development, production and dissemination phases, especially given the activities planned. Key departures during the development and production cycle increased the level of stress for the team and the risk for program delivery. For example, during interviews, it was noted that when a CEAG team member left unexpectedly, their tasks were held up because there was no one available to immediately fill in for the missing resource.

The recruitment of junior and senior validators was delayed because of challenges in securing a sufficient number of qualified validators for the 2021 CEAG.^{Footnote 13} As a result, the recruitment efforts had to use initiatives that were not pre-planned or pre-approved (e.g., exemption from language policy for senior validators). According to the lessons learned document, the planning and resourcing of the dissemination activity cycle occurred too late, making it challenging to take full advantage of the resources.

3. Performance—communication strategy and activities

Summary

Overall, the 2021 CEAG communication strategy and activities were effective in maintaining, and in some cases improving, awareness of CEAG products. New and improved tools increased awareness and facilitated users' understanding of how to interpret CEAG data. Some areas noted for consideration for subsequent cycles of the CEAG are continuing to amplify the CEAG through various communication and engagement channels; empowering users to access, use and interpret the data; and providing clearer information and guidance on changes.

The 2021 CEAG communication and dissemination strategy supported the achievement of high-quality data that are accessible, available and relevant.

CEAG communications prior to Census Day and throughout the collection period contributed to data quality and efficiency by promoting participation through messaging about the benefits of completing the questionnaire and methods used to reduce response burden (e.g., data replacement). Diversifying the communication channels during collection and dissemination, including social media campaigns, the census website and the FPT Committee on Agriculture Statistics, helped to reach the various stakeholder groups. External interviewees who were members of the committee reported being satisfied with the regular CEAG communication updates at their quarterly meetings.

Communication and dissemination efforts also contributed to the accessibility and availability of data through the release of tools and guidance materials before the official data release in May 2022, a new online portal housing all CEAG-related content and publications, new interactive data visualization and mapping tools, and webinars highlighting CEAG products. The dissemination activities also supported data relevance through the publication of analytical reports focusing on cross-cutting themes and emerging sectors.

The COVID-19 pandemic led to the transition of all promotional activities to online engagement, but most external key interviewees did not perceive this transition to have limited promotion and awareness-raising activities, as more people transitioned online. However, a few external interviewees noted that additional methods to increase awareness of the CEAG and its products may be beneficial (e.g., farm shows, radio spots, promotional materials and direct emails).

The 2021 CEAG also used a combination of articles, infographics, and new interactive data visualization and mapping tools to support end users' engagement with the data. Microdata were also made available to researchers at the research data centres. Data users reported that making data available using a variety of methods, especially the infographics and visualization tools, improved data access and use.

The communication strategy and dissemination activities were effective in maintaining, and in some cases improving, awareness of CEAG products.

The majority of external interviewees reported that their awareness of CEAG products was similar to the previous census cycle, because they are often returning to the same products after every census cycle. However, some data users reported being made aware of the new data visualization and mapping tools, and the timeline of releases. Although web analytics data for the 2021 CEAG could not be compared with those from 2016 because of software changes within Statistics Canada, the uptake of the mapping and data visualization tools was relatively stable from month to month, indicating steady and ongoing use.

End users often reported The Daily as the main source of CEAG-related updates, releases and news, and the new CEAG portal as the primary access point to CEAG data. However, users also noted that they would use Google to search for relevant data, as opposed to going through the official Statistics Canada website. A recommendation to improve awareness of CEAG products was to email stakeholder groups a week or two prior to a product's release.

Overall, most data users reported being satisfied with the accessibility of CEAG data. The majority of data users accessed CEAG data directly through the new CEAG portal, which, for the most part, was considered user friendly. While the portal was revised based on early stakeholder consultation, a few data users described challenges finding the relevant data on the portal. In addition to the online portal, some external interviewees accessed CEAG data through custom data requests via email.

End users' understanding of how to use CEAG tools was largely maintained or improved as a result of the new user guide and webinars. Their understanding of how to interpret CEAG data was supported by data visualization tools, infographics and articles.

Most external interviewees reported that their understanding of how to use the CEAG tools remained the same, largely as a result of being long-time users of CEAG data. The publication of the user guide improved understanding of some of the changes to the 2021 CEAG and some of the new analytical tools that were introduced. Although the methodological changes were presented to various partners throughout the planning cycle (e.g., FPT Committee on Agriculture Statistics), a few external interviewees reported that the CEAG could do more to engage with data users early on to help them understand the methodological changes and how they will affect their analysis and interpretation of CEAG data.

Webinars also supported data users' understanding of how to use the CEAG tools. However, there are still opportunities to improve data users' understanding of how to use the tools (e.g., tables), particularly for those who are inexperienced with CEAG data. There was also a request for webinars teaching experienced data users how to use CEAG tools for more advanced analytical purposes.

Most data users reported that their understanding of how to interpret CEAG data remained similar to that for the 2016 CEAG. The continued publication of infographics and the new data visualization tools, along with the analytical articles, were all perceived to enhance understanding and use of CEAG data. A few external interviewees reported a desire for more analytical reports, including sector-specific reports that were available in past CEAG cycles.

Communication strategies and activities to consider for the next CEAG cycle include continuing to amplify and target CEAG promotion and data through social media and other means, for both engagement and dissemination activities, and empowering new and advanced end users to access, use and interpret CEAG data.

CEAG social media engagement was viewed positively by end users and internal interviewees. End users supported the continued use of social media to engage stakeholders and suggested that platform-specific campaigns be considered, depending on the target audience (e.g., X, formerly known as Twitter, was reported to be used more by producers).

Interviewees mentioned that the provision of webinars or other educational tools was useful and noted that tailoring them to levels of expertise (e.g., novice, advanced) would further support CEAG tool usage and interpretation. Additionally, some users commented that there is an opportunity to increase end users' awareness and understanding of what informational needs custom data requests can help fulfill, and the process involved in making a request.

Transitioning to publications that transcend all sectors may be undertaken in conjunction with more end-user education on how to find, use and interpret publicly available or customized data products.

The dissemination plan outlines the transition to a more sector-wide, horizontal approach to publications that are relevant to a wider audience. However, eliminating or limiting commodity-specific reports may need to be partnered with more support to end users to increase their knowledge and skills on how to find, use and interpret data, especially for smaller commodity sectors.

How to improve the program

Proper planning includes identifying, assessing and addressing risks and challenges, and helps to ensure that adequate time and resources are available for a program to deliver on its priorities and commitments to achieve its outcomes. About half of the interviewees noted that there were gaps in planning and resources for the 2021 CEAG, which led to significant pressures on staff. Given the challenges around staffing and the loss of experience and expertise resulting from departures, human resource planning will be critical for subsequent cycles. In addition, the large-scale changes implemented for the 2021 CEAG have fundamentally altered many processes, establishing a new baseline moving forward. The 2026 CEAG will provide an opportunity to verify that the planned resource levels and allocations match this new baseline.

Past initiatives to raise awareness of products and tools have been effective and well received; the CEAG should continue these efforts. Amplifying and targeting communication and engagement efforts will help increase understanding of CEAG products and tools, the intentions and goals of new methodologies and collection vehicles, and the corresponding benefits, thereby supporting further acceptance.

Recommendation 1

The ACS, Economic Statistics (Field 5), should

• ensure that the process in place to regularly review resources includes revisiting the balance between upcoming and planned deliverables (including those for communication and engagement) and the corresponding challenges versus available time and resources
• in addition to the regular review process, given that the 2026 CEAG is not expected to include any new large-scale changes, verify that the planned levels and allocation of resources are correctly aligned with the new baseline resulting from the changes made for the 2021 CEAG; this will help ensure that future core activities are resourced appropriately.

Recommendation 2

The ACS, Economic Statistics (Field 5), should ensure that effective training, retention and succession plans are developed, reviewed regularly and aligned with future needs of the CEAG program.

Recommendation 3

The ACS, Economic Statistics (Field 5), should ensure that the CEAG has a communication and engagement plan in place that includes

• varied communication approaches to broaden reach and clearly communicate the CEAG's intentions and plans, including the corresponding benefits, to increase stakeholders' understanding and acceptance
• adequate approaches to improve data users' awareness of and engagement with CEAG data, to further support CEAG tool usage and data interpretation, including working with users with less statistical capacity.

Management response and action plan

Recommendation 1

The ACS, Economic Statistics (Field 5), should

1. ensure that the process in place to regularly review resources includes revisiting the balance between upcoming and planned deliverables (including those for communication and engagement) and the corresponding challenges versus available time and resources
2. in addition to the regular review process, given that the 2026 CEAG is not expected to include any new large-scale changes, verify that the planned levels and allocation of resources are correctly aligned with the new baseline resulting from the changes made for the 2021 CEAG; this will help ensure that future core activities are resourced appropriately.

Management response

Management agrees with the recommendation.

As a large project, the CEAG currently follows the agency's Departmental Project Management Office (DPMO) planning and reporting procedures and will continue to do so. In addition, the team will conduct a review of its 2026 Census project planning processes, in consultation with the DPMO, to find improvements.

Given the substantial changes that took place during the 2021 Census, the team will ensure that its planning and reporting procedures reflect the new reality to confirm that the expected workload associated with deliverables assigned to a given team does not exceed its resource availability.

In addition to finalizing a 2026 project plan, the CEAG team will add integrated scheduling to its existing planning and also carefully review resources and deliverables using the DPMO dashboard as a key coordination vehicle. The integrated schedule will ensure that the deliverables for both subject-matter and service areas are aligned with the new 2026 baseline and that all activities are resourced appropriately.

The project plan will be regularly monitored using the DPMO monthly dashboard and monthly CASC approvals.

Deliverables and timelines

A project plan approved by the CASC will be available by October 2024, and an integrated schedule, also approved by the CASC, will be available by January 2026.

Recommendation 2

The ACS, Economic Statistics (Field 5), should ensure that effective training, retention and succession plans are developed, reviewed regularly and aligned with future needs of the CEAG program.

Management response

Management agrees with the recommendation.

The CEAG team will review its existing training and documentation activities and, in collaboration with the agency's Workforce and Workplace Branch, create a human resources plan, including training, retention and succession planning activities.

The plan will be reviewed regularly and include the establishment of partnerships with the agency's Workforce and Workplace Branch to ensure that staffing activities occur as required and seek exemptions from policies or directives that may impede human resources activities, e.g., determinate versus indeterminate status. Where possible, the CEAG team will leverage activities within the CePop program.

The plan will also focus on

• attracting and retaining high-performing employees
• developing more robust succession planning to stabilize transitions in times of staff turnover
• implementing practices to improve the integration of new employees, e.g., improving documentation.

Deliverables and timelines

A human resources plan approved by the CASC will be delivered by September 2025.

Recommendation 3

The ACS, Economic Statistics (Field 5), should ensure that the CEAG has a communication and engagement plan in place that includes

1. varied communication approaches to broaden reach and clearly communicate the CEAG's intentions and plans, including the corresponding benefits, to increase stakeholders' understanding and acceptance
2. adequate approaches to improve data users' awareness of and engagement with CEAG data, to further support CEAG tool usage and data interpretation, including working with users with less statistical capacity.

Management response

Management agrees with the recommendation.

1. The CEAG developed a communication and engagement plan for the 2021 cycle, which was approved by the CASC. Within budget constraints, the CEAG will work with partners in Census Communications to review and revamp this plan to create a new 2026 plan, in alignment with the AgZero initiative and its associated benefits. The new plan will explore varied communication approaches to broaden reach and outline the CEAG's benefits to increase stakeholders' and farmers' understanding and acceptance.
2. The 2021 CEAG developed a comprehensive dissemination plan with a vision to continue to meet user needs: more cross-cutting, horizontal stories and more outreach with Canadians via various media. The plan outlined key areas to enhance the CEAG data user experience: data products, data visualization products and mapping tools, analytical products, user guides, and webinars. Similarly, within the approved budget, a new enhanced 2026 dissemination plan will be created in collaboration with communication and dissemination service areas and be approved by the CASC.

A key focus of the plan will be to articulate approaches to improve data users' awareness of and engagement around CEAG data to further support CEAG tool usage and data interpretation, including working with users with less statistical capacity. These approaches could include giving targeted webinars, sending emails and using social media to announce releases, and consulting with users to better meet their needs.

Deliverables and timelines

A 2026 CEAG communications and engagement plan and a 2026 CEAG dissemination plan, developed in collaboration with relevant service areas and both approved by the CASC, will be delivered by September 2025 and December 2026, respectively.

Appendix A – Interview quantification scale

Interview responses are quantified and categorized in this report using the scale shown in the table below.

Appendix A – Examples of data needs on intersectionality

Term	Definition
One	One is used when one participant provided the answer.
Few	Few is used when 4% to 15% of participants responded with similar answers. The sentiment of the response was articulated by these participants but not by other participants.
Some	Some is used when 16% to 45% of participants responded with similar answers.
About half	About half is used when 46% to 55% of participants responded with similar answers.
Most or a majority	Most, or a majority, is used when 56% to 89% of participants responded with similar answers.
Almost all	Almost all is used when 90% to 99% of participants responded with similar answers.
All	All is used when 100% of participants responded with similar answers.