November 2023
Project number: 80590-127

Table of contents

• Executive summary
• Conformance with professional standards
• Introduction
  • Background
  • Audit objective
  • Scope
  • Approach and methodology
  • Authority
• Findings, recommendations and management response
  • Hiring managers' views on staffing
  • Staffing advisor and human resources function views on staffing
  • Monitoring the efficiency and effectiveness of the staffing process
  • Recommendations
• Appendices
  • Appendix A: Audit criteria
  • Appendix B: Acronyms

Executive summary

People are Statistics Canada's most valuable resource. Collectively, they drive the agency's mandate and priorities in producing statistics that help Canadians better understand their country. Staffing processes must be effective and efficient to ensure Statistics Canada hires the right people with the right competencies at the right time.

Staffing processes are a shared responsibility between the hiring manager and the staffing advisor. The hiring manager is responsible for making selection decisions that respect the public service values and ethics while acting in conformity with organizational goals. The staffing advisor is responsible for providing options, strategic advice, and guidance on the available staffing mechanisms and associated risks and considerations. The human resources (HR) assistant works alongside their assigned staffing advisor by being responsible for administrative components of the staffing process, such as creating and maintaining the staffing action file in GCdocs. Finally, Corporate Staffing is responsible for designing the staffing process; monitoring; and overseeing many of the fundamental components that support staffing advisors and HR assistants in executing staffing actions, such as developing tools and templates, developing training and guidance, and representing the agency in the event of a complaint.

The agency has experienced an extraordinary staffing landscape since the onset of the COVID-19 pandemic . Annual staffing actions processed by the agency grew from 6,473 in 2019/2020 to 13,947 in 2021/2022, before falling to 8,643 in 2022/2023. As hiring ramped up to meet the demands of new projects and programs, staffing processes were strained, leading to a backlog of staffing actions. This sudden increase took place during a period when HR leadership and teams were seized with efforts to transition the agency to full telework (followed by a shift to a hybrid model), multiple implementations of vaccine reporting, the hiring of thousands of contact tracers and the implementation of a rapid testing program for census employees. However, through significant work and effort, and without additional resources, the agency successfully recruited all required employees to work on the new projects and programs. This extraordinary staffing volume is not expected to be repeated in the foreseeable future.

Why is this important?

Staffing actions more than doubled over the last few years, leading to a persistent backlog and increasing the length of time to hire. An effective and efficient staffing process is critical to ensuring Statistics Canada has the HR required to achieve its mandate and priorities.

Further, according to the Public Service Commission's Appointment Framework, Statistics Canada is required to conduct a cyclical assessment at least once every five years. This assessment takes a broad look at the health of the organizational staffing system and identifies areas that need to be strengthened, as well as potential measures to address weaknesses. This Audit of Staffing contributes to fulfilling this requirement.

Overall conclusion

The agency's staffing processes have been stressed by a sudden increase in the volume of staffing transactions, combined with the departure of experienced staffing advisors. To optimize the effectiveness and efficiency of the staffing process as the agency moves towards more regular staffing levels, opportunities for improvement were identified in the areas of staffing advisor support and development, process design and documentation, staffing integrated systems, and the monitoring of and reporting on the process.

Key findings

Hiring managers' views on staffing

While the agency has been successful in recruiting employees to meet the increased demand in recent years, the administrative burden of staffing processes on hiring managers has resulted in some frustration among hiring managers. They find it cumbersome and overly compliance driven, inefficient, and sometimes ineffective in finding the best candidate. While acknowledging the challenges faced by staffing advisors, hiring managers would like to better leverage HR expertise throughout the staffing process.

Staffing advisor and human resources function views on staffing

There has been significant turnover in the staffing team in recent years. Most staffing advisors are new to their role, yet do not feel adequately supported. Staffing processes are not well defined, are not optimized at the agency level, are inadequately supported by integrated information systems and undergo frequent change. Together, these factors add administrative burden for staffing advisors and make it difficult for them to fully meet hiring manager expectations.

Monitoring the efficiency and effectiveness of the staffing process

There is limited reporting on metrics related to the efficiency and effectiveness of staffing processes overall. This type of monitoring would support data-driven decision making and the ongoing improvement of the effectiveness and efficiency of the staffing process.

Conformance with professional standards

The audit was conducted in accordance with the Mandatory Procedures for Internal Auditing in the Government of Canada, which include the Institute of Internal Auditors'International Standards for the Professional Practice of Internal Auditing.

Sufficient and appropriate audit procedures have been conducted, and evidence has been gathered to support the accuracy of the findings and conclusions in this report and to provide an audit level of assurance. The findings and conclusions are based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria. The findings and conclusions are applicable to the entity examined and for the scope and period covered by the audit.

Steven McRoberts
Chief Audit and Evaluation Executive

Introduction

Background

People are Statistics Canada's most valuable resource. Collectively, they drive the agency's mandate and priorities in producing statistics that help Canadians better understand their country. Staffing processes must be effective and efficient to ensure Statistics Canada hires the right people with the right competencies at the right time.

The agency has defined three specific objectives for the staffing process that can be summarized as follows:

• Achieving operational objectives: The agency must build an agile and competent workforce through effective human resources (HR) planning activities and staffing and recruitment strategies adapted to its organizational context and business needs.
• Compliance: The agency must ensure compliance with the Public Service Employment Act (PSEA), the Values and Ethics Code for the Public Sector, and requirements of internal and central agency policies.
• Equity, diversity and inclusion: Hiring managers should always look for the best qualified person to fill a position. They must also put in place strategies to attract talent from different backgrounds who offer a wide variety of ideas and strengths that, together, lead to innovative outcomes.

Staffing processes are a shared responsibility between the hiring manager and the staffing advisor. The hiring manager is responsible for planning for their workforce; identifying needs; engaging meaningfully with HR throughout the process; providing information as required; and ultimately making selection decisions that respect public service values and ethics, legislation, and policies while acting in conformity with organizational goals. The staffing advisor is responsible for providing options, strategic advice, and guidance on the available staffing mechanisms and associated risks and considerations. The HR assistant works alongside their assigned staffing advisor by being responsible for administrative components of the staffing process, such as creating and maintaining the staffing action file in GCdocs. Finally, Corporate Staffing is responsible for designing the staffing process; monitoring; and overseeing many of the fundamental components that support staffing advisors and HR assistants in executing staffing actions, such as developing tools and templates, developing training and guidance, and representing the agency in the event of a complaint.

Staffing mechanisms within the federal public service are subject to the PSEA, the Treasury Board's Policy on People Management, and the Public Service Commission's (PSC) Appointment Framework. Notably, the PSC's Appointment Framework was launched in 2016 with the intention of streamlining requirements, reducing administrative burden, offering greater flexibility and accountability, encouraging agile and customized approaches to staffing and policies, and increasing focus on outcomes. Within Statistics Canada, hiring also adheres to the agency's Staffing Governance Framework, which took effect on May 15, 2022.

In 2020/2021, like all organizations, Statistics Canada responded to the COVID-19 pandemic quickly, pivoting operations to focus on mission-critical programs and, with an unprecedented need for data, delivering data-driven insights to Canadians at a time when they were needed most. Throughout this time, the agency experienced an extraordinary staffing landscape as annual staffing actions processed by the agency grew from 6,473 in 2019/2020 to 13,947 in 2021/2022, before falling to 8,643 in 2022/2023. This sudden increase took place during a period when HR leadership and teams were seized with efforts to transition to full telework (some 7,500 employees were moved to telework, a great advance), followed by a shift to a hybrid model (including the use of personas, pulse surveys and an employee wellness survey to support employees' transition to a hybrid work environment); multiple implementations of vaccine reporting; the hiring of thousands of contact tracers (over 2,000 additional Statistical Survey Operations employees were hired in less than four months); and the implementation of a rapid testing program for census employees. As hiring ramped up to meet the demands of new projects and programs, the agency struggled to keep pace, leading to a persistent backlog of staffing actions. However, through significant work and effort, and without additional resources, the agency successfully recruited all required employees to work on the new projects and programs. This extraordinary staffing volume is not expected to be repeated in the foreseeable future.

Audit objective

The objective of this audit is to provide reasonable assurance on the effectiveness and efficiency of staffing processes and tools to facilitate appropriate and timely staffing of personnel.

Scope

The scope of the engagement included an examination of selected components of the agency's management control framework for staffing, including processes to

• ensure management and the Workforce and Workplace Branch work in partnership to execute on staffing needs and priorities
• enable all parties to understand and adhere to their respective staffing responsibilities
• enable effective and timely staffing in accordance with management's needs.

Compliance with the PSC's Appointment Framework was excluded from the engagement scope, as the Workforce and Workplace Branch conducts annual self-assessments of staffing files to assess compliance and identify areas for improvement.

For the purposes of this engagement, the staffing process was considered to begin when the hiring manager identifies a need to fill a vacancy and to be complete once the letter of offer is issued to the selected candidate.

The scope period covered processes in place for the fiscal years 2021/2022 to 2022/2023.

Executive staffing was excluded from the engagement scope, as it follows different processes and requirements from other staffing actions.

Finally, the engagement scope also excluded the following elements, as they would be better assessed through separate, targeted engagements: the parts of the hiring process that take place after the selection of the successful candidate (including the security clearance process and pay-related processes); equity, diversity and inclusion initiatives; official languages measures; and financial management controls.

Approach and methodology

This audit was conducted in accordance with the Mandatory Procedures for Internal Auditing in the Government of Canada, which include the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing.

The audit work consisted of

• examination and analysis of relevant internal documentation
• questionnaires for hiring managers and staffing advisors
• interviews and walkthroughs with hiring managers from across the agency, staffing operations team leads, staffing advisors, HR assistants and a team lead, along with representatives from Corporate Staffing and HR management.

Authority

The audit was conducted under the authority of the approved Statistics Canada Integrated Risk-based Audit and Evaluation Plan (2023/2024 to 2027/2028).

Findings, recommendations and management response

Hiring managers' views on staffing

While the agency has been successful in recruiting employees to meet the increased demand in recent years, the administrative burden of staffing processes on hiring managers has resulted in some frustration among hiring managers. They find it cumbersome and overly compliance driven, inefficient, and sometimes ineffective in finding the best candidate. While acknowledging the challenges faced by staffing advisors, hiring managers would like to better leverage HR expertise throughout the staffing process.

Staffing is a shared responsibility between hiring managers, staffing advisors and HR assistants. Staffing advisors and HR assistants support the administration of the process, while hiring managers are accountable for planning their staffing needs, making selection decisions, and ensuring compliance with staffing-related legislation and policies.

Staffing processes are cumbersome and overly compliance driven

While the agency's objectives for staffing do not explicitly include the concept of efficiency, achieving the three objectives in the least amount of time and at the lowest cost to the agency remains an important, yet unstated, goal. Further, these objectives are sometimes in tension with one another, requiring careful balancing to optimize results. Specifically, achieving operational objectives and ensuring compliance are often at odds, as attaining compliance adds burden to the process and can delay staffing actions, putting operational objectives at risk.

Hiring managers noted that staffing templates, established to demonstrate compliance, are burdensome and time-consuming to complete. They questioned whether the agency was striking the right balance between operational and compliance objectives, noting that often the feedback they received when completing the forms added marginal business value. Hiring managers also reported that staffing advisors each had their own interpretation of what was required and the point at which compliance requirements were satisfied.

Completing the Candidate Assessment Validation (CAV) form was identified as a common challenge in this regard. This form exists to fulfill compliance objectives and the PSEA requirement that the hiring decision for non-advertised appointments be documented. Staffing advisors and hiring managers alike noted that hiring managers spend significant time in completing the document and that it often included multiple rounds of going back and forth between the hiring manager and staffing advisor. The CAV template did not provide detailed guidance on what could represent sufficient documentation of the hiring decision, leaving room for varying interpretations of the point at which the documentation was acceptable.  

Striving for perfection in achieving compliance can lead to inefficiencies and unnecessary complications. While it is essential to meet the requirements set by the PSC, aiming for perfection beyond the established threshold can result in prolonged staffing processes, increased administrative burden and cost, and delays in filling critical positions. An overemphasis on perfection can also divert resources and attention from other pressing matters, leading to an imbalance in priorities.

Certain staffing processes are slow and may not result in finding good candidates

Two-thirds of hiring managers noted that traditional staffing processes and assessments were not always effective in finding the right candidate. While candidates often looked good on paper and answered questions appropriately in the interview stage, their on-the-job performance was not always reflective of their assessment results. Certain types of staffing actions are very labour-intensive for hiring managers, with varying results in finding candidates with the specific skills they are looking for. Other challenges included processes receiving too few applicants and taking too long (applicants often accepted positions elsewhere when the processes took too long).

Hiring managers also reported using several staffing actions to expedite employee start dates with the agency. In such scenarios, once a candidate is selected, they are offered an acting appointment to allow time for an indeterminate appointment to be processed. While this practice effectively speeds hiring, it also increases the workload for both staffing advisors and hiring managers as it creates two staffing actions in the place of one.

Siloed human resources functions create extra steps in the process for hiring managers

There are various HR functions that can be involved in a hiring process, including staffing and classification, along with other functions within the Corporate Strategy and Management Field (e.g., security, official languages and finance). Hiring managers reported taking on the facilitator role between the various HR functions, which were viewed as siloed. Several hiring managers noted that they are uncertain which group to reach out to for various HR issues, and this could add to the complexity and time of the staffing process. Members of the staffing function also expressed similar concerns.

Hiring managers want to better leverage strategic support from their staffing advisors

While acknowledging the challenges faced by staffing advisors, hiring managers indicated they would like staffing advisors to take on more responsibility throughout the staffing process. This would allow the agency to leverage the expertise of the staffing advisors and would align with the desired evolution of the staffing function. This change would include support such as providing strategic advice, assisting with screening large numbers of candidates and helping with the completion of mandatory compliance documentation. As the staffing function is not currently resourced or equipped to provide this level of support, some fields and divisions have created staffing "shadow shops" to perform these tasks. Using their knowledge of the hiring manager's business needs and past knowledge of the staffing process, these support teams act as a liaison between hiring managers and staffing advisors.

However, these shadow shops create challenges for staffing advisors. They disconnect the staffing advisor from the business, resulting in the advisor focusing mostly on the administrative aspects. Further, the guidance provided by the shadow shops may not align with that of the staffing advisor. While some of these shops have been disbanded or reduced, the demand for their services has not waned. Many staffing advisors acknowledged the need for the services these shadow shops provide. The team of staffing advisors wants to better support clients and acknowledged the challenges in the current environment, which include the high volume of staffing actions, the number and diverse needs of hiring managers they serve, and the administrative burdens caused by a manual and inefficient staffing process (discussed further in the Monitoring the efficiency and effectiveness of the staffing process section below).

The staffing process has not been designed to optimize efficiency and effectiveness

Many of the issues identified point towards a staffing process that has been designed based on accountabilities, rather than efficiency and effectiveness. Along these lines, the total cost of the staffing process is not known or monitored. While the cost of the staffing function itself is known, the considerable time invested by hiring managers in staffing is not tracked. Hiring manager time has thus become an unmeasured resource when designing the staffing process, and over time the process can drift towards being optimized at the staffing function level rather than the agency level. This results in tasks being assigned to hiring managers because they are accountable for them, not because they are the most efficient or effective people to perform them. In many cases, tasks for which hiring managers are accountable may be better or more efficiently performed by a centralized resource with the knowledge and volume of transactions to become proficient in the tasks.

For example, hiring managers are accountable for the selection decision. Consequently, they are expected to define the competencies required for a position, develop assessment tools for each competency, and evaluate the assessment tools for bias—all with limited support or training in these activities. Each of these activities can benefit from in-depth knowledge and experience and may be more effectively and efficiently performed, or guided, by a centralized resource with expertise in competencies and their assessment. Under an agency-level mindset, a more efficient and effective process might have a centralized resource maintain a bank of competencies, complete with definitions and bias-free assessment tools. Hiring managers could leverage these as required, rather than having to independently research competencies and develop assessment tools for their own personal use. While the agency does maintain a bank of tools for larger collective processes that are repeated on a periodic basis, similar support is not available for individual processes.

Staffing advisor and human resources function views on staffing

There has been significant turnover in the staffing team in recent years. Most staffing advisors are new to their role, yet do not feel adequately supported. Staffing processes are not well defined, are not optimized at the agency level, are inadequately supported by integrated information systems and undergo frequent change. Together, these factors add administrative burden for staffing advisors and make it difficult for them to fully meet hiring manager expectations.

Staffing advisors, operating within the existing manually driven processes, are faced with many challenges while supporting hiring managers. Many staffing advisors are new to their role and would like more support and the opportunity to learn from more experienced advisors. A commonly identified theme was that staffing advisors have the best interests of the agency and hiring managers in mind and are dedicated to providing an effective service.

Most staffing advisors are new to their role

Staffing advisors play a key role in the staffing process. They are process experts, responsible for guiding hiring managers from need identification to the appointment of the chosen candidate. They provide strategic advice on staffing mechanisms, risks and compliance requirements. This advice demands a strong understanding of their clients' business and staffing needs; of how to reach, attract and assess high-quality candidates; and of how to navigate the complex web of regulations, policies and directives that govern federal government hiring.

Across the federal government, staffing advisors are in short supply and in high demand, making movement between departments a common challenge.

As noted, the agency has seen a sharp increase in the volume of staffing actions over the past few years. Concurrently, turnover rates among the agency's Personnel Administration (PE) category have also spiked. During this time, the agency experienced turnover rates of 25% to 33% among PEs, compared with 7% to 12% across the federal public service. In 2022/2023, the staffing function had a turnover rate of 36%, compared with a turnover rate of 23% for the agency.

Management reported difficulty in replacing departing experienced staffing advisors with similarly experienced advisors. Consequently, junior staffing advisors (PE-01 or PE-02 levels) were hired in their place. As of September 2023, the agency had 14 staffing advisors in total: 12 junior staffing advisors at the PE-01 or PE-02 level and 2 senior staffing advisors at the PE-03 level.  

Staffing advisors require more support, straining team leads working to fill the gap

Developing a staffing advisor takes time, training and mentoring. They begin at the PE-01 level as a developmental HR advisor, typically for a period of one year or more, before progressing to the PE-02 level as an HR advisor, at which point they deliver operational services of limited scope. Junior staffing advisors are supported throughout this process by a more experienced staffing advisor at the PE-03 (HR specialist) or PE-04 (team lead) level. This advisor serves as a mentor, allowing the junior advisor to shadow them, reviewing their work before it is communicated to the hiring manager and acting as a sounding board when they encounter difficult situations. It is only once staffing advisors have attained the PE-03 HR specialist level that they may work fully independently and are considered to provide expert staffing advice.

Because of this high turnover among senior staffing advisors, junior staffing advisors have been assigned as lead advisors to work directly with hiring managers without an assigned mentor (but with supervision from their team lead). In some respects, they are performing the responsibilities of a senior staffing advisor without having the knowledge, experience or support to perform at that level. Junior staffing advisors reported taking on roles they do not feel sufficiently equipped or supported to execute. Many reported being uncomfortable providing advice to senior managers and executives without having experience with all types of staffing actions (particularly advertised processes) or fully understanding staffing processes.

With a shortage of experienced staffing advisors, team leads are struggling to fill the gap. The staffing team is presently composed of two team leads, each of whom is assigned seven to eight staffing advisors. Staffing management reported that team sizes of four to five were more typical, with a mix of junior and experienced advisors. Team leads reported being overwhelmed with their workload and having to work overtime to keep up while providing coaching, review and ongoing support to junior staffing advisors. Staffing advisors also noted that team leads seemed overburdened, making staffing advisors hesitant to approach them when they needed help. They stated this has led to delays in responding to hiring managers and has slowed the staffing process.

Frequent rotation of staffing advisors across client groups is causing frustration for hiring managers and staffing advisors

Adding to these challenges, the high turnover and volume of staffing actions have resulted in a frequent rotation of client groups among staffing advisors. Hiring managers and staffing advisors expressed frustration with this constant change and noted that greater stability was required. For staffing advisors, frequent rotation makes it difficult to establish strong working relationships with their clients and to understand their business and their needs. For hiring managers, a new staffing advisor requires dedicating time to update them on ongoing staffing actions, their business and their broader staffing needs. It also means adjusting to different ways of doing things, which can lead to challenges as advisors may have different interpretations of what is required and how the staffing process should proceed.

The shortage of experienced staffing advisors able to provide expert advice, coupled with the frequent rotation of staffing advisors across client groups, has affected the quality of service staffing advisors can provide and the cohesion between hiring managers and staffing advisors. These factors also serve to compound the other challenges discussed throughout this report.

Hiring managers are not involving staffing advisors early in the process

HR management noted that a common challenge concerned staffing advisors not being informed of staffing needs and activities early enough in the process. Last-minute requests result in backdated transactions and a high volume of urgent staffing actions. These issues make it difficult for staffing advisors to plan their work; complicate pay processes; and create problems for employees, who risk being paid wrong, late or not at all, or experiencing delays or refusal of benefits. These issues also affect the relationship between hiring managers and staffing advisors. Hiring managers expressed a desire for more support from their staffing advisors. Likewise, staffing advisors expressed a desire to provide more strategic advice and support to hiring managers and to be seen as a trusted partner in their clients' staffing activities. However, not involving staffing advisors early in the process relegates them to simply processing paperwork, often under tight time pressures. This in turn can lead to staffing advisors being seen as an obstacle to hiring managers, as the staffing advisor is left scrambling to ensure all the administrative requirements necessary to process the staffing action are fulfilled.

The agency introduced a timeliness initiative in May 2023 to establish clear timelines that hiring managers must meet for HR actions to be processed in time. As the initiative was implemented during the audit, it was not possible to assess its effectiveness. That said, management hopes that it will increase rigour and discipline in staffing processes and lead to hiring managers including staffing advisors earlier in their staffing actions.

Staffing processes are not well defined, making it difficult to optimize the process

Well-designed and implemented processes and procedures contribute to an effective staffing function. They help ensure common and standardized approaches in the administration of staffing actions, establish clear expectations for the level of service provided by staffing advisors, and inform all participants of the requirements and timelines of a typical staffing action. They can also ensure the most efficient approach is taken on each staffing action by properly sequencing actions and ensuring that no steps are forgotten.

Staffing advisors and hiring managers noted that the process was not fully defined. The HR function has not developed clear, comprehensive and documented procedures outlining the specific steps required for each type of staffing process. That said, a comprehensive flowchart was developed for the non-advertised staffing process in coordination with Field 7's Operations and Integration Division but was not shared with staffing advisors or hiring managers as the project was discontinued because of fiscal constraints. This work is a good start and could be leveraged in documenting other types of staffing actions. Various checklists have also been developed to define the key documentation required for each type of staffing action, but hiring managers and junior staffing advisors reported that these were not sufficiently detailed to meet their needs as they do not include all the steps of the process, the expected timelines for each step or the purpose behind each step.

The lack of clear, standardized procedures has resulted in a process that varies depending on the staffing advisor and hiring manager involved.

As an example, a key step at the outset of a staffing action is the development of the statement of merit criteria (SoMC). This document outlines the experience, skills and competencies^{Footnote 1} required for the position and forms the basis of the candidate assessment. Completing the SoMC is the responsibility of the hiring manager, with guidance from the staffing advisor. There was no common approach to determining the level of support that should be provided. Also, staffing advisors who reported sharing SoMCs from other departments were sharing them from their personal collection rather than a list curated and maintained by the agency and available to all other staffing advisors.

Non-integrated information systems create administrative burden for staffing advisors

Information management and information technology (IT) systems are required to facilitate efficient and effective staffing processes. These systems should support process flows, document management, and monitoring and reporting.

Statistics Canada employs several systems to support its staffing process. These systems include Orbit, the Staffing Activity Management System, a work-in-progress (WIP) spreadsheet shared among all staffing advisors, SharePoint for financial approvals, GCdocs for key documents, staffing advisor and HR assistant mailboxes for correspondence and documents in progress, and various spreadsheets and other unique tools created and used by staffing advisors and HR assistants to track their files. Notably, these systems are not integrated, affecting the availability and accuracy of data related to the staffing process. Many data points are duplicated across multiple systems, leading to conflicting information that needs to be reconciled when information is compiled for reporting purposes.

These disparate systems add significant burden for staffing advisors, create complications and frustration for hiring managers, cause delays to staffing processes, and take staffing advisor time away from providing strategic support to their clients. There are also multiple entry points for new staffing actions, meaning staffing advisors must monitor their emails, SharePoint and WIP to know what work they have underway. Reporting for senior management requires staffing advisors to manually compile and reconcile data from multiple systems. Hiring managers cannot see the status of their staffing actions, leading to numerous emails to staffing advisors requesting status updates that must also be manually compiled and reconciled by staffing advisors. Additionally, the systems do not support the process of completing the required forms, meaning that they must be shared by email between the hiring manager, their delegate and the staffing advisor during their development, leading to lost documents and version control issues. These issues create frustration and undermine the hiring manager's trust in the staffing advisor in several ways. First, the lack of system-level support for the staffing process can make it hard for staffing advisors to locate files from previous staffing actions, giving the appearance of disorganization to hiring managers. Second, staffing advisors sometimes ask hiring managers to provide documents a second or third time for a given process, causing frustration and a loss of trust among hiring managers who understood that the staffing advisor was responsible for storing documentation for the staffing action.

Further, there is no formal documentation on which system is used to track what information, who should enter the information and when it should be done. Staffing advisors and their HR assistants work out roles and responsibilities between themselves. Consequently, some HR assistants may do more to support their staffing advisor while others may do less. These inconsistencies are manifested in the information available in the systems.

The HR team has undertaken a project to implement an integrated system to automate monitoring and improve the efficiency of the staffing process. However, this is not its first attempt to do so. Previous attempts have been set aside in favour of newer or more urgent priorities or have had their scope changed during implementation such that they did not deliver the integrated system that had been envisioned. Several concerns were raised regarding this current implementation that warrant consideration:

• Senior HR management stated an expectation of implementation in the fall at minimal cost (attributable to having the selected software available to the agency at no additional cost). However, according to IT staff, a project of this scope will take months (at minimum) to implement, making a fall implementation unrealistic. They further noted that the development costs will be significant.
• Two competing software alternatives are being considered. These alternatives were chosen for their low cost (the agency already owns licensing for them) and pre-existing in-house support. However, the system selection is being conducted while user needs are still being defined by Corporate Staffing, and staffing processes have not yet been documented.
• The requisite resources from IT and Human Resources Business Intelligence (HRBI) have not been formally assigned to the project. It is being conducted "from the corner of the desk," and the HRBI team expressed concerns about competing priorities and deadlines that may affect the system implementation. The availability of the IT team to work on this project is also dependent on other priorities and resource constraints.

Frequent process changes with insufficient communication affect awareness and buy-in

Staffing advisors, Corporate Staffing and hiring managers reported a high pace of change in staffing priorities and processes. Staffing advisors are asked to communicate these changes to hiring managers and enforce the changes when hiring managers push back. However, staffing advisors said they were frequently not consulted on the changes or their implementation or told why the change was being made. As a result, there are missed opportunities to consult staffing advisors for their input and to prepare them to answer questions from hiring managers and build buy-in and awareness for the changes.

Monitoring the efficiency and effectiveness of the staffing process

There is limited reporting on metrics related to the efficiency and effectiveness of staffing processes overall. This type of monitoring would support data-driven decision making and the ongoing improvement of the effectiveness and efficiency of the staffing process.

Establishing clear performance targets and tracking the performance of the staffing process are important and can provide valuable data to support the improvement of its efficiency and effectiveness.

The agency has established timeliness service standards for each type of staffing action, but these standards are insufficient for assessing the efficiency of the overall process. These standards address only the time elapsed after all the documentation has been satisfactorily completed by the hiring manager. It therefore excludes the time spent planning the staffing action and soliciting, screening, interviewing and assessing candidates, as well as the time spent by the hiring manager documenting the selection decision. It also excludes any time spent going back and forth with the staffing advisor to bring the documentation to a satisfactory state.

Without clear targets and more complete monitoring of the time to hire (from need identification to start date), cost to hire (inclusive of hiring manager time) and time spent on the various steps of the process, it is difficult to know whether the staffing process is appropriately designed to meet agency needs or to understand where it may be falling short. Including hiring manager time—by tracking it through the Time Management System or elsewhere—in the cost of the process will be critical to understanding whether portions of the process would be better assigned to lower-cost resources or centralized resources with specialized expertise. These targets, particularly time to hire and cost to hire, should be negotiated with senior management to ensure that the staffing process is designed to appropriately balance cost, compliance objectives and business needs.

The effectiveness of the process is also not currently monitored. This can include monitoring the effectiveness of specific aspects of the staffing process (such as quality of staffing advisor advice and guidance) but should also include monitoring the quality of hires. Doing so could help identify which types of staffing processes are most successful, as well as areas where more work is required to improve effectiveness, such as better identifying the skills and competencies required for a job or designing new assessment tools that more accurately predict on-the-job performance.

Recommendations

Recommendation 1

It is recommended that the assistant chief statistician, Corporate Strategy and Management, ensure that

1. a consistent approach to each type of staffing action be documented, communicated and implemented.

Management response

Management agrees with the recommendation.

A comprehensive review and analysis of staffing processes will be undertaken, prioritizing the staffing actions that account for the greatest staffing volume, and will include

• working with partners in fields 6 and 7 to define and document each type of staffing process, including the steps within the process, the timelines associated with each step, and the roles and responsibilities of each individual involved
• producing documentation and tools that support a standardized approach to staffing processes, with clear roles and responsibilities for each individual involved and associated timelines
• refreshing the staffing page on the Internal Communications Network (ICN) to articulate staffing processes, steps and requirements, as well as roles and responsibilities for each individual involved in the process.

Deliverables and timeline

The director general, Workforce and Workplace Branch, will

1. document process maps for each type of staffing action (approximately 20 in total), 10 of which account for 96% of the actions that make up the staffing transaction volume; the first 10 processes will be done in 2024, and the remaining lower-priority processes will be done in 2025, with the following schedule: 5 processes documented by the end of June 2024, 5 processes documented by the end of December 2024, 5 processes documented by the end of June 2025 and the remaining processes documented by December 2025
2. launch tools spanning guides, infographics and training materials by the beginning of June 2024, with the release of process-specific tools aligning with timeframes in the previous deliverable
3. refresh and update the staffing page on the ICN by the beginning of June 2024.

Recommendation 2

It is recommended that the assistant chief statistician, Corporate Strategy and Management, ensure that

2. a plan to address gaps in the development, day-to-day support and retention of staffing advisors be developed and implemented.

Management response

Management agrees with the recommendation.

A thorough review will be conducted to understand and implement the measures required to better support the development and retention of staffing advisors.

To address the findings of the review, the following will be developed: standard procedures; supporting tools; and a development program for staffing advisors that encompasses a blend of formal training, informal training, coaching, and exposure to a range of staffing files and actions to support development and consistency in service delivery. This includes reviewing and updating the PE Recruitment and Development Program, support measures, and training roadmaps and career paths for staffing advisors.

Deliverables and timeline

The director general, Workforce and Workplace Branch, will

1. document a review of planned measures to support the development and retention of staffing advisors by September 2024.

Recommendation 3

It is recommended that the assistant chief statistician, Corporate Strategy and Management, ensure that

3. a business case be developed and executed for the implementation of an integrated system to enable the staffing function to efficiently carry out its responsibilities, reduce administrative burden and enable monitoring.

Management response

Management partially accepts the recommendation.

Because of austerity, it is not anticipated that any investment in an integrated system will be supported by central agencies over the next number of years. Management will

• conduct an internal scan to determine integrated system needs, gaps and immediate options
• conduct an external scan with the Office of the Chief Human Resources Officer to determine anticipated implementation dates for Next Generation Human Resources and Pay (NextGen HR and Pay) to determine potential implementation at Statistics Canada
• as an interim measure, implement Workbench to improve tracking of actions and improve oversight of processes.

Deliverables and timeline

The director general, Workforce and Workplace Branch, will

1. present a documented internal and external scan to define integrated system needs, gaps, and recommended short- and long-term solutions (including NextGen HR and Pay) for the approval of the assistant chief statistician, Corporate Strategy and Management, by March 2024
2. complete Workbench system changes to improve tracking of actions and improve oversight of processes by June 2024. 

Recommendation 4

It is recommended that the assistant chief statistician, Corporate Strategy and Management, ensure that

4. key performance indicators and targets for the efficiency and effectiveness of the staffing process be developed, implemented, monitored and reported to senior management.

Management response

Management agrees with the recommendation.

Management will establish a systematic approach to measure the performance and health of the staffing activity. This will include developing quantitative key performance indicators (KPIs) and targets to gauge the efficiency of staffing processes (including time elapsed and effort expended on staffing actions) and the effectiveness of the staffing process (including measures of hiring manager satisfaction).

Deliverables and timeline

The director general, Workforce and Workplace Branch, will

1. present to the Operations Committee for review and approval a plan for the monitoring and reporting of staffing activity results, including the proposed monitoring cycle by June 2024
2. present to the Operations Committee for review and approval specific KPIs and targets to be monitored by December 2024 (for the first 10 staffing processes documented, as outlined in the first deliverable)
3. present reporting of results to the Operations Committee by December 2025 (and ongoing).

Recommendation 5

It is recommended that the assistant chief statistician, Corporate Strategy and Management, ensure that

5. an agency-level plan to assess and improve the efficiency and effectiveness of the staffing process be developed, following the implementation of actions associated with recommendations 1 through 4, in a way that balances operational objectives with compliance requirements; senior management should be consulted in its development and approve the plan, and progress on its implementation should be reported to a tier 1 governance committee periodically.

Management response

Management agrees with the recommendation.

Following implementation of actions linked to recommendations 1 through 4, management will conduct an annual review of the staffing system to further identify efficiencies, review compliance requirements and measure effectiveness of the function, with input from senior management.

Deliverables and timeline

The director general, Workforce and Workplace Branch, will

1. present to the Operations Committee for approval a review of the efficiency and effectiveness of the staffing function and proposed action plan to address areas for improvement, the initial part of the review to begin in June 2026, following the completion of process and efficiency reviews of all staffing processes, with a target presentation date of December 2026.

Appendices

Appendix A: Audit criteria

Audit criteria

Control objectives	Core controls and criteria	Policy instruments and sources
1. The planning of staffing processes is efficient and effective, and hiring managers, staffing advisors and human resources (HR) assistants are adequately supported.	1.1 The planning of staffing processes is efficient. 1.2 The planning of staffing processes is effective in defining the staffing need and establishing a staffing strategy that fulfills the identified need. 1.3 Hiring managers, staffing advisors and HR assistants are adequately supported throughout the planning process.	Statistics Canada's Staffing Governance Framework Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors Human resource planning is aligned with strategic and business planning. (PPL-1) The organization provides employees with the necessary training, tools, resources and information to support the discharge of their responsibilities. (PPL-4) Suitable policies and procedures to support the development and management of human resources are established, maintained and communicated. (PPL-7)
2. The assessment of candidates is efficient and effective, and hiring managers, staffing advisors and HR assistants are adequately supported.	2.1 The assessment of candidates is efficient. 2.2 The assessment process is effective in determining candidate suitability. 2.3 Hiring managers, staffing advisors and HR assistants are adequately supported throughout the assessment process.	Statistics Canada's Staffing Governance Framework Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors Human resource planning is aligned with strategic and business planning. (PPL-1) The organization provides employees with the necessary training, tools, resources and information to support the discharge of their responsibilities. (PPL-4) Suitable policies and procedures to support the development and management of human resources are established, maintained and communicated. (PPL-7)
3. The selection of candidates is efficient and effective, and hiring managers, staffing advisors and HR assistants are adequately supported.	3.1 The selection process is efficient. 3.2 The selection process is effective in facilitating the hiring manager's candidate selection decision. 3.3 Hiring managers, staffing advisors and HR assistants are adequately supported throughout the selection process.	Statistics Canada's Staffing Governance Framework Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors Human resource planning is aligned with strategic and business planning. (PPL-1) The organization provides employees with the necessary training, tools, resources and information to support the discharge of their responsibilities. (PPL-4) Suitable policies and procedures to support the development and management of human resources are established, maintained and communicated. (PPL-7)
4. The staffing process is monitored to support its continuous improvement and ensure business needs are met.	4.1 Targets to monitor the efficiency and effectiveness of the staffing process are established, aligned with business needs and monitored. 4.2 Results of process monitoring are used to identify and address areas of weakness.	Statistics Canada's Staffing Governance Framework Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors The oversight body / bodies request and receive sufficient, complete, timely and accurate information. (G-6) Management has identified planned results linked to organizational objectives. (RP-1) Management has identified appropriate performance measures linked to planned results. (RP-2) Management monitors actual performance against planned results and adjusts course as needed. (RP-3)

Appendix B: Acronyms

CAV

Candidate Assessment Validation

HR

Human resources

HRBI

Human Resources Business Intelligence

ICN

Internal Communications Network

IT

Information technology

KPI

Key performance indicator

NextGen HR and Pay

Next Generation Human Resources and Pay

PE

Personnel Administration

PSC

Public Service Commission

PSEA

Public Service Employment Act

SoMC

Statement of Merit Criteria

WIP

Work-in-progress

Environment, traditional knowledge, and territory

• Inuit Nunangat comprises roughly 40% of Canada's land area and 72% of its coastline. In 2021, there were 70,545 Inuit living in Canada, with just over two-thirds (69.0%) living in Inuit Nunangat. (First Nations people, Métis and Inuit in Canada)
• In 2017, 33% of First Nations people living off reserve, 35% of Métis, and 65% of Inuit had hunted, fished, or trapped. (Harvesting activities among First Nations people living off reserve, Métis and Inuit: Time trends, barriers and associated factors)
• In 2021, while Indigenous people represented 5% of the total population, they represented 12.4% and 12.9% of residents of fishing and forestry resource-based communities. (Selected population characteristics of people living in resource-based communities, by resource industry)
• Since 2016, the Indigenous farm population, which comprises farm operators and the individuals in their households, has risen 6.0%, reaching 16,705 people in 2021. (The socioeconomic portrait of the Indigenous farm population in Canada, 2021)

Children and youth

• The number of elementary and high school students who participated in programs where an Indigenous language is taught as a subject (as part of the regular course offerings) increased nationally from 53,913 in 2017/2018 to 59,355 in 2021/2022. (Number of students in Indigenous language programs, public elementary and secondary schools, by program type)
• In 2017, most Indigenous youth reported that they felt good about their Indigenous identity: 91% of First Nations youth living off reserve, 93% of Métis youth, and 97% of Inuit youth. (Portrait of Youth in Canada: Data Report, Chapter 4: Indigenous Youth in Canada)

Languages, cultures, and art

• While the number of Indigenous people reporting an Indigenous language as their mother tongue declined by 7.1% between 2016 and 2021 censuses, the number of those who learned an Indigenous language as a second language continued to rise. In 2021, over one-quarter (27.7%) of Indigenous language speakers learned it as their second language, up from 24.8% in 2016. (Indigenous languages across Canada)
• Gross domestic income earned by Indigenous workers and Indigenous-led businesses in the arts, entertainment and recreation sector as well as the information and cultural industries sector exceeded $1.1 billion in 2021. (Gross domestic income attributable to Indigenous people by industry)

Women, girls and 2SLGBTQ+ people

• In 2022, the gender pay gap was slightly smaller among Indigenous people than non-Indigenous people, though Indigenous people still earned 9% less than non-Indigenous people. (Quality of employment among First Nations people living off reserve and Métis, 2022)
• In the 2021/2022 academic year, women represented 69% of Indigenous new entrants to undergraduate degree programs, compared with 59% of non-Indigenous students. (Highlights on Indigenous new entrants to postsecondary education)
• Between 2019 and 2021, Indigenous people who identified as 2SLGBTQ+ made up 7.0% of the Canadian 2SLGBTQ+ population, even though Indigenous people overall made up 5.0% of the total Canadian population. (Socioeconomic characteristics of the 2SLGBTQ+ population, 2019 to 2021)

Quality of life statistics

• Indigenous people were less likely to report having a good or great deal of confidence in Canadian institutions when compared with non-Indigenous people, such as the police (48% versus 63%), the justice system (30% versus 47%) and federal Parliament (16% versus 33%). (Confidence in Canadian Institutions)

Health care access

• From 2017 to 2020, significantly higher proportions of First Nations people living off reserve (20.3%), Métis (17.9%) and Inuit (56.5%) reported being without a primary health care provider, compared with their non-Indigenous counterparts (14.5%). (Primary health care access among First Nations people living off reserve, Métis and Inuit, 2017 to 2020)
• Between 2017 and 2020, roughly one-half of First Nations people living off reserve (47.6%) and Métis (51.4%) reported using a doctor's office as their usual place of care for minor health problems, followed by walk-in clinics. Among Inuit, one-fifth (21.2%) reported using a doctor's office while about double that proportion (41.8%) used community health centres. (Primary health care access among First Nations people living off reserve, Métis and Inuit, 2017 to 2020)

Education and employment

• Involuntary part-time work (less than 30 hours per week) was significantly higher among Indigenous people (23.9%) than among non-Indigenous people (18.6%) in 2022. This rate was much higher among Indigenous men than women (31.3% versus 20.8%). (Quality of employment among First Nations people living off reserve and Métis, 2022)
• In 2022, First Nations people living off reserve and Métis had higher rates of short tenure jobs (less than 12 months) than non-Indigenous people (24.3% and 21.6% versus 19.1%). (Quality of employment among First Nations people living off reserve and Métis, 2022)
• When comparing 2021 with five years earlier, the share with a bachelor's degree or higher increased for First Nations people (+1.6 percentage points (p.p.)), Métis (+2.5 p.p.) and Inuit (+0.9 p.p.). (Postsecondary educational attainment and labour market outcomes among Indigenous peoples in Canada, findings from the 2021 Census)
• Although the overall employment rate among Indigenous adults in 2021 was lower than the non-Indigenous population (61.2% vs 74.1%), the difference disappears when only considering those with a bachelor's degree or higher (82.6% and 82.5%, respectively). (Postsecondary educational attainment and labour market outcomes among Indigenous peoples in Canada, findings from the 2021 Census)

The NetSupport software is required for our organization to be able to provide an effective Quality Control Monitoring Program. It will enable supervisors to provide comprehensive feedback to data collection clerks on core competencies in areas that cannot otherwise be addressed via auditory observation alone. This assessment illustrates that the risks involved with using NetSupport are far outweighed by the crucial function it provides. The application is essential for providing the necessary training, support and coaching data collection clerks need to meet our high standards for data quality control.

Objective

A privacy impact assessment was conducted to determine if there were any privacy, confidentiality and security issues associated with using NetSupport for the Quality Control Monitoring Program, and if so, to make recommendations for their resolution or mitigation.

Description

Statistics Canada has a legislative mandate under the Statistics Act to collect survey information from respondents on various topics. To fulfill this mandate, the agency is responsible for the objectives of maintaining high standards for quality control and conducting effective performance management of employees. The current Quality Control Monitoring Program was established in the regional offices in pursuit of achieving these objectives, however the program is rendered ineffective by the absence of visual observation for telephone interviews.

The NetSupport software is a third-party application that will address the need to have visual observation during monitoring sessions of telephone interviews. For our usage, the application will allow a set of users (i.e., Data Collection Supervisors) to conduct visual observation of other users' desktop screens (i.e., data collection clerks) for the sole purpose of performance management and quality control. The software will enable supervisors to evaluate in real-time the data collection clerk's skills, ensure data quality and identify areas for improvement in the interviewing process. As a result, the quality and level of detail supervisors can provide to data collection clerks for their performance feedback is greatly enhanced. At no point will the information viewed via NetSupport be recorded, collected or saved by the software.

For privacy impacts related to NetSupport, measures are in place to ensure that the impacted parties are informed. Data collection clerks are made aware of the possibility of being observed via their collective agreement, the training they receive and the interview script they read. Respondents of Statistics Canada surveys are made aware of monitoring at the beginning of the interview via the standard statement in our survey scripts that indicates a supervisor may listen to the call for quality control purposes.

Although NetSupport will not be saving/recording information, users of the software will have access to view the respondent's personal information while it is visible on the data collection clerks' screen. This may contain personally identifiable information such as names, addresses, ages, and other demographical information, as well as responses to our survey questions. Given the confidential nature of this information, the use of NetSupport will be restricted to supervisors and managers who have taken the Oath of Secrecy. These employees receive training on how to handle and protect confidential information that is subject to the Privacy Act and the Statistics Act.

Our organization is taking the necessary precautions to limit the number of risks involved with using NetSupport for respondents and employees. For example, the use of NetSupport will be limited to observing work-related performance only and for limited durations (e.g., 20 minutes). The user settings for NetSupport will also be restricted so that features that go beyond the purposes of monitoring performance will be disabled (e.g., recording of audio or video) to ensure it meets our security procedures. Furthermore, permissions for user accounts will be standardized and aligned with internal security procedures.

Risk Area Identification and Categorization

The PIA identifies the level of potential risk (level 1 is the lowest level of potential risk and level 4 is the highest) associated with the following risk areas:

a) Type of program or activity Risk scale
Administration of program or activity and services	2
b) Type of personal information involved and context
Only personal information, with no contextual sensitivities, collected directly from the individual or provided with the consent of the individual for disclosure under an authorized program.	1
c) Program or activity partners and private sector involvement
Within the institution (among one or more programs within the same institution)	1
d) Duration of the program or activity
Long-term program or activity.	3
e) Program population
The program's use of personal information for internal administrative purposes affects certain employees.	1
f) Personal information transmission
The personal information is used in a system that has connections to at least one other system.	2
g) Technology and privacy
The NetSupport software will be used to support the existing quality control and performance management program of the regional offices. It will assist supervisors with fulfilling their responsibility of monitoring telephone interviews and provide performance feedback. It does not collect, create or handle personal information. Aside from the request for the NetSupport Manager application itself, no new electronic systems or applications are needed to support the program in terms of creation, collection or handling of personal information. The application will need to be integrated into our networks to allow users to view the activity of the data collection clerks. This will require some modifications to our current firewalls in order to ensure this connection is established. Shared Services Canada (SSC) and Statistics Canada's IT partners have conducted preliminary testing and it has been confirmed that integration is feasible. The NetSupport software will involve surveillance as an administrative tool to measure the performance (via visual observation) of Statistics Canada's data collection clerks. As the information that may be accessed through NetSupport may potentially be personally sensitive for our respondents, it could be perceived as surveillance of respondents as there will be an additional person viewing their responses as they are submitted. To address this concern, the respondents will be made aware during the telephone call that a supervisor may be listening for the purpose of quality control. It is important to note that the surveillance purposes of the program are restricted to improving internal quality control practices only. For example, the NetSupport application will be used to observe if the data collection clerk is following procedures, entering data correctly and using appropriate techniques such as probing. The responses provided in a questionnaire by the respondent are not particularly relevant to the user who is accessing the information. They are a by-product of the quality control work, and not the main focus. Irrespective of the relevance of the data, all confidential information that is observed via NetSupport will be protected under the Statistics Act Oath of Office that is taken by the NetSupport user and the guidelines of the Values and Ethics Code. This protection applies to the data collection clerk's activities as well as the respondent's participation.
h) Potential risk that in the event of a privacy breach, there will be an impact on the individual or employee.
The potential risk of a privacy breach involving our organization's use of NetSupport is very low. The impact of a privacy breach on the individual or employee will be very limited due to the measures in place. Statistics Canada has legislation and practices that reduce and mitigate the potential risks involved in handling confidential information. Data Collection Supervisors will be using the NetSupport software for visual observation while conducting a monitoring session of the data collection clerk's telephone interview. These supervisors are deemed employees who have sworn an Oath of Office to protect the confidential information they encounter during their work activities. For their jobs, they handle and encounter confidential information on a daily basis and must adhere to the guidelines and repercussions of the Statistics Act and the Privacy Act. Statistics Canada is taking precautionary measures to reduce the impacts for individuals and employees regarding the use of NetSupport. For example, we are disabling features within the application that would pose additional privacy risks, such as recording, so that it cannot save or retain information. Also, the application will solely be used for the purpose of monitoring work activities of data collection clerks and helping supervisors conduct performance management. These restrictions provide an accountability framework for limiting the possibility for confidential information to be disclosed improperly. Safeguards are also in place to prevent the risk of a privacy breach coming from unauthorized access to Net Support. In addition to the existing security measures for preventing outside access to our networks, the application will be customized for enhanced security. This will involve requirements for user accounts, as well as passwords, and two-factor authentication, further limiting the group of people who may have access to the application.
i) Potential risk that in the event of a privacy breach, there will be an impact on the institution.
The potential risk for a privacy breach involving NetSupport to impact the institution will be very limited due to the measures in place. The practices and safeguards for using NetSupport will offer reasonable layers of accountability in the event that confidential information viewed via NetSupport is improperly disclosed. This organizational infrastructure will allow for a breach to be dealt with efficiently in a manner that reduces the impact on the institution. For example, the regional offices will be responsible for managing users of the program and ensuring their work activities adhere to the requirements of the Statistics Act. The checks and balances that exist within the reporting structure of the regions will help to ensure that users are held accountable for their use of NetSupport. In the event of wrongful disclosure of confidential information, users of NetSupport will be subject to the same recourse as for activities undertaken for general statistical programs (see Generic Privacy Impact Assessment for Statistics Canada's Statistical Programs.) . Furthermore, since the application will not be used to retain or store data there is a reduced risk for mishandling of information.

Conclusion

This assessment of NetSupport did not identify any privacy risks that cannot be managed using existing safeguards.

Date: March 2024

Program manager: Director General, Census Management Office

Reference to Personal Information Bank (PIB):

Personal information collected through the Census Test is described in Statistics Canada's "Census of Population - Census Program Content Test" Personal Information Bank (Bank number: StatCan PPU 007) which collects information related to individuals who have participated in content tests conducted to test Census of Population Program questions.

The bank describes individuals within households selected to participate in Census Program test surveys. Personal information may include name, contact information, demographic information (including age, date of birth, ethnicity, and religion), Social Insurance Number (SIN), citizenship status, languages spoken or understood, relationship to cohabitant(s), educational information, financial information, work and other social and economic characteristics depending on the particular census or survey.

The "Census of Population - Census Program Content Test" PIB (Bank number: StatCan PPU 007) is published on the Statistics Canada website under the latest Information about Programs and Information Holdings chapter.

Description of statistical activity:

Under the authority of the Statistics Act^Footnote1, Statistics Canada is conducting the mandatory Census Test from April to June 2024. The objectives of the Census Test which involves a sample of dwellings across the country are to determine whether new or revised questions under consideration for the 2026 Census of Population can be easily understood and correctly answered, to assess public reaction to these questions by testing them on a smaller scale basis, and to evaluate behaviour of staff when using new systems and procedures.

The Census of Population's purpose is to provide statistical information, analyses and services that measure changes in the Canadian population and, demographic characteristics. It serves as a basis for public and private decision making, research and analysis in areas of concern to the people of Canada.

The Census of Population is a reliable basis for the estimation of the population of the provinces, territories and municipal areas. The information collected is related to federal and provincial legislative measures and provides a basis for the distribution of federal transfer payments. The census also provides information about the characteristics of the population and its housing within small geographic areas and for small population groups to support planning, administration, policy development and evaluation activities of governments at all levels, as well as data users in the private sector.

Access to trusted statistical information is an essential foundation of democratic societies, both to support evidence-based decision-making in the public and private sectors and to inform debate on public policy issues. Particularly in today’s fast-paced environment, with an unprecedented pace and scope of change, accurate statistical information, comparable over time and across various levels of geography, is essential to help Canadians better understand their country in the context of timely, relevant and high-quality statistical information on Canada’s changing economy and society.

To ensure the continued relevance of the census to Canadians, Statistics Canada conducts a formal consultation at the start of each census cycle. During that time, Statistics Canada invites data users, stakeholders and the general public to provide feedback on what information they use, for what purpose and what, if any, data gaps Statistics Canada should consider addressing in the next census cycle. A Consultation Report, 2026 Census of Population Consultation Results: What we heard from Canadians, will be published on Statistics Canada's website in April 2024^Footnote2.

As in past censuses, extensive consultations on the questions to include in the 2026 Census of Population have been held with Canadians. New and modified questions, developed to reflect new needs identified in the consultations, have been qualitatively tested by Statistics Canada in 2023 using methods such as individual interviews and discussion groups.

Reason for supplement:

While the Generic Privacy Impact Assessment (PIA) addresses privacy and security risks related to statistical activities conducted by Statistics Canada, this supplement describes potential new risks associated with the collection, processing and use of data related to new census content being considered, and the possible concerns of Canadians about the intrusiveness of the collection. As is the case with all PIAs, Statistics Canada's privacy framework ensures that elements of privacy protection and privacy controls are documented and applied.

Statistics Canada uses its Content Determination Framework process to identify potential new census content for inclusion in the Census. The process begins by consulting external stakeholders on their uses of the data and related information needs, including requirements for relevance and quality. Statistics Canada also assesses the Canadian context which encompasses respondent burden and societal privacy concerns, and evaluates other considerations such as costs, operational factors, safeguards against loss of data quality and safeguards against loss of efficiency and/or quality in other Statistics Canada programs. The suitability of alternative sources (i.e., administrative data) is also considered as part of this process.

Key content additions for the 2024 Census Test are:

• Household composition and marital status
  • Modifications have been made to the format of the household composition and marital status module to include new questions and changes to the response options and question wording to reduce respondent burden, use inclusive language, and to better reflect changes in Canada’s social context.
• Second address
  • A two-part question has been added to identify people who live or stay at more than one address, including children in joint custody or students who live elsewhere while studying, to improve measures of family dynamics and living arrangements and to better enumerate fluctuating populations, important for municipal and urban planning.
• General health
  • A new question on general health has been included to collect information on the health status of people in Canada, including vulnerable populations and those living in small areas.
• Homelessness
  • A set of three new questions has been included to capture Canadians’ experiences with homelessness and other housing challenges over the past 12 months and to identify people currently in precarious housing situations. These data aim to help inform governments and organizations on levels of homelessness and other housing challenges particularly for rural and hard to reach communities.
• Commuting
  • A new question has been included to collect more detailed data on which days workers typically commute to work to better understand changes in commuting frequencies as many workplaces shift to hybrid models or remote work.
• Childcare expenses
  • The question has been restructured to ask about the amount paid for childcare per child younger than 12 years in the household and to identify any unpaid childcare. To enhance its analytical and policy relevance in the early learning and childcare domain, the scope of the question was changed to include childcare for reasons beyond enabling parents or guardians to work.

Necessity and Proportionality

The collection and use of personal information for the 2024 Census of Population Test can be justified against Statistics Canada’s Necessity and Proportionality Framework:

1. Necessity: Statistics Canada's mandate is to ensure that Canadians have access to a trusted source of statistics that meet their highest priority information needs. The efficient production of relevant, accessible, high-quality statistics helps to ensure that the Canadian economy functions efficiently and our society is governed effectively. As part of this mandate, Statistics Canada is responsible under the Statistics Act (R.S.C., 1985, c. S-19) for conducting the Census of Population every five years. By law, the government (by an Order-in-Council) prescribes the questions to be asked in the census. By the same law, each person is required to provide the information requested in the census and Statistics Canada must protect the confidentiality of the personal information provided by respondents. The next census will be conducted in May 2026. Being a major statistical operation, the census presents a formidable challenge for Statistics Canada and requires careful planning to ensure its successful completion. An essential element to ensuring this success is the conduct of a census test, which traditionally is scheduled two years prior to the census. The content proposed for the census is the result of a Census Content Determination Framework that begins with consulting external stakeholders on their uses of the data and related information needs including the requirements for relevance and quality, followed by an assessment of the Canadian context which encompasses respondent burden and societal privacy concerns, and an assessment of Statistics Canada considerations including costs, operational factors, data quality and safeguards against loss of efficiency and/or quality in other Statistics Canada programs.

   The specific content under consideration responds to high priority data requirements identified through consultation, for example:

   • Household composition and marital status
     • The family and household data resulting from this question are essential for all levels of government to develop effective policies and legislation regarding the composition of families and households. Data help identify, plan, evaluate and administer federal, provincial and local programs for childcare, schools, support for people with disabilities, older adults living alone, one-parent families and other vulnerable populations.
   • Second address
     • The purpose of this question is to fill a data gap for a variety of census data users (urban planners, municipalities, provincial and federal departments) who require accurate population counts in their regions in order to better plan the delivery of services and programs. This information would also reflect family dynamics and living arrangements, for example, by enumerating populations who spend time at multiple addresses, including children in joint custody or students who live elsewhere while studying.
   • General health
     • General health assessments are unavailable from population-level administrative data sources and although general health assessments are commonly included in health surveys, those surveys have limited sample sizes and population coverage. Including a general health question in the Census would provide necessary health information for key specific and/or vulnerable populations, which can assist in health care planning and policies. General health assessments are unavailable for the entire population which has resulted in significant data gaps for key vulnerable populations (e.g., First Nations people living on reserves, individuals with disabilities, and children and youth living in unique family structures).
   • Homelessness
     • These data will help fulfill reporting requirements under the National Housing Strategy Act, which recognizes housing as a human right and commits the federal government to further the progressive realization of the right to adequate housing.
   • Commuting
     • The Census is the only cross-national source of place of work and journey to work data. It is heavily used by provincial and municipal governments across Canada, in addition to researchers within various organizations.
     • Place of work and journey to work data support numerous federal, provincial, and municipal programs related to infrastructure and public transportation, as well as such things as sustainable development and environmental goals.
   • Childcare expenses
     • Childcare has recently emerged as a critical social policy in Canada. In December 2022, Bill C-35, An Act respecting early learning and child care in Canada, (the Canada Early Learning and Childcare Act)^Footnote3 was introduced in the House of Commons. Bill C-35 declares that the Government of Canada has as a goal to support the establishment and maintenance of a Canada-wide early learning and childcare system where families of varying incomes have access to affordable, inclusive and high-quality early learning and childcare programs and services regardless of where they live.
     • Along with the income data obtained from personal income tax and benefit records through data integration, these questions help determine the disposable income for Canada’s official measure of poverty—the Market Basket Measure.

   Note also that the 2024 Census of Population Test will collect data on sex at birth and gender, which was introduced in the 2021 Census so that more Canadians could see themselves in their census data, and to meet new data needs and adapt to societal changes. It allowed for the dissemination of information on transgender and non-binary populations. The historical question on “sex” was modified in 2021 to ask about “sex at birth”, for added precision.

   StatCan has assessed the Gender-based Analysis Plus implications with respect to the 2026 Census of Population cycle. The Census is representative of Canada’s entire population and data will be collected and presented in a way that allows for gender-based analysis. Data will be presented separately for males and females and by other identity characteristics. Prior to 2016, the census collected sex with binary options of male or female. In the 2016 Census, respondents who could not respond in a binary fashion were asked to leave their answer blank and provide a comment at the end of the questionnaire. After consulting with LGBTQ2+ organizations and conducting focus groups with transgender, non-binary and cisgender individuals, the 2021 Census collected sex at birth and gender. In preparation for the 2026 Census, Statistics Canada will consult with LGBTQ2+ organizations. The results of these discussions will contribute to content determination for the 2026 Census.

   More information on how Statistics Canada balances the protection of confidentiality with the needs for disaggregated census data is available in the Age, Sex at Birth and Gender Reference Guide, Census of Population, 2021.

2. Effectiveness – Working assumptions: The collection and use of this data during the 2024 Census Test will be effective in meeting Statistics Canada's objectives because it is necessary to ensure that the 2026 Census of Population produces relevant, high priority, statistically meaningful information.

   Many of the content changes proposed for the 2026 Census and that will be tested in 2024, affect smaller population groups such as hybrid workers, children in joint custody and students, Veterans, visible minorities, Canadians in housing need, Indigenous populations etc. To be effective, the content test needs to accurately discern statistical differences between 2021 content (benchmark) and modified content to ensure that proposed content can produce relevant and statistically meaningful information.

3. Proportionality: Data from the Census of Population Program provide a comprehensive socioeconomic portrait of Canada's population over time that supports key requirements for policy areas, including labour market conditions, immigration, Indigenous peoples, education, mobility, skills development, official languages, housing and income. This information is vital to all levels of government, the private sector, academia, and non-profit organizations for decision-making and for developing and monitoring of programs and policies. To ensure the efficient and successful conduct of the 2026 Census of Population, Statistics Canada is conducting the 2024 Census Test, which will comprise a sample of approximately 198,000 households across Canada. The test serves multiple purposes, from testing improved or new questions to testing improvements to collection tools and procedures. The sample size and the complex test design were determined in such a way as to ensure that all the test objectives are met, while minimizing as much as possible the respondent burden and the cost. Smaller qualitative tests were previously conducted to help minimize the sample size of the 2024 Census Test. The test is the perfect opportunity to gather data that supports the analysis of the impacts of proposed content changes on data quality, response burden, etc. The data provides a solid foundation upon which Statistics Canada’s final recommendations for census content changes are based, and upon which Cabinet can base their decision on the final content for the next census cycle.
4. Alternatives: The Census of Population is one of the only sources of information for small geographic areas based on the same statistical concepts for the entire country, and the only source of information for many socioeconomic characteristics. The most efficient method for effectively testing possible changes to the Census questionnaire is the mandatory sample survey to be conducted in May 2024.

   Prior to conducting the 2026 Census, extensive testing will be done to assess changes to content, communication materials, field procedures, and collection methodologies. The 2024 Test is designed to accurately measure respondent behaviours to changes in content, field and collection methodologies. Mandatory census tests provide the most accurate impact measurement when studying small sub-groups of the population.

   Voluntary tests in 2024 would increase the chance of inaccurate or inconclusive findings for many of the proposed changes to questionnaire content. If responding to the test was voluntary, non-response bias could occur, specifically for smaller population groups. Similar concerns were raised in the context of the 2011 National Household Survey (NHS). The 2011 NHS demonstrated that in a voluntary context, some respondents tend to skip questions or abandon more often, amplifying non-response for certain questions, especially those closer to the end of the questionnaire. These questions could be more impacted, leading to wrong conclusions being drawn, and, as a result, recommendations for new or changed content for the 2026 Census being based on incomplete test results. In addition, without reducing the risk of non-response bias, a voluntary test would require a larger sample size to account for total non-response. Based on a preliminary design, a significant number of additional dwellings would need to be part of the test, and with anticipated higher proportion of non-response, there would be an increase in the number of reminders and contact attempts for selected households.

   Further, although Statistics Canada continues to explore the use of new administrative sources to replace data collection from respondents, the research has shown the sources of administrative data currently available are not adequate to replace collection of the new content under consideration for the 2026 Census of Population.

Mitigation factors:

The 2024 Census Test will adhere to all Statistics Canada policies on collection, processing, and dissemination of information.

The overall risk of harm to the survey respondents has been deemed manageable with existing Statistics Canada safeguards that are described in Statistics Canada’s Generic Privacy Impact Assessment, including the following that relate to this specific activity.

Storage

Respondents' personal information is transmitted directly to Statistics Canada via secure, web-enabled electronic questionnaires, or, in the case of paper questionnaire formats, through the mail directly to a Statistics Canada secured processing site, where the information is captured, and an electronic record created.

Records are stored in a response database containing household responses, including personal information to perform record linkages to the 2021 Census of Population response files and to tax files.

All electronic records are stored within the secure Statistics Canada information technology environment. Paper questionnaires are securely stored within the Statistics Canada access-controlled processing environment.

Records are retained for a period of five years after the completion of the Census Test and then are destroyed.

As well, Statistics Canada complies with the Policy on Government Security and the Policy on Service and Digital.

Processing and analysis

The Agency's longstanding linkage experience has shown that particular combinations of personal information elements can be used to identify individuals in different data sources with a very high level of confidence, and thus link individuals across various files.

The following personal information will be collected and used to perform record linkages:

• First and last names (collected)
• Date of birth (collected)
• Sex/Gender (collected)
• Full civic address (from frame and collected)
• Phone number (from frame and collected)
• Mobility one and five years (collected)
• Country of citizenship (collected)
• Place of birth (collected)
• Immigrant status (collected for respondents getting Control questionnaire or Test questionnaire version 2; added through linkages for respondents getting Test questionnaire version 1)
• Year of immigration (collected for respondents getting Control questionnaire or Test questionnaire version 2; added through linkages for respondents getting Test questionnaire version 1)
• Frame identifier (from frame)
• Relationship to Person 1 (collected)
• Marital Status (collected)
• Common-law Status (collected)

The linkage of Census of Population Program data and 2024 Census Test data will be used in statistical analyses to evaluate the impact of new or modified questions of the questionnaire by comparing the 2024 Census Test to other existing sources or to the data collected in the 2021 Census. The linkage file will be used only within Statistics Canada, for methodological research, development, and processing. No statistical estimates will be published from the linked data.

Personal identifiers such as name, phone number and address (excluding postal code) will be removed from linked files and replaced with an anonymized statistical identifier.

Access

Access to any confidential data held by Statistics Canada is closely monitored and restricted to designated individuals as per the program's operational requirements. Employees are required to provide the justification for access and obtain the necessary approval. Furthermore, all access permissions are only applicable for a set duration of time and must be regularly renewed including justification for re-approval.

The response database for the 2024 test has not yet been created. The number of employees with access to the 2024 Census Test response database will vary depending on the timing of 2026 Census preparation activities and the need for staff to have access for their job function. Access to the identifying information (names, address, and phone numbers) within this database will be controlled and limited to a small set of staff requiring such access. Most staff who will analyze the data will not have access to any identifying information within the database.

Dissemination

Data collected during the 2024 Census Test will be used for Statistics Canada's analytical purposes only and will not be publicly disseminated. Statistics Canada will not disclose personal information from the 2024 Census Test without the consent of the respondent.

Openness

It is the policy of Statistics Canada to provide all respondents with information about: the purpose of a survey (including the expected uses and users of the statistics to be produced from the survey), the authority under which the survey is taken, the mandatory or voluntary nature of the survey, confidentiality protection, the record linkage plans and the identity of the parties to any agreements for sharing of the information provided by those respondents, where applicable.

For the 2024 Census Test, this information is provided in the letter of invitation to complete the Census Test questionnaire, in the electronic questionnaire itself and in Frequently Asked Questions (FAQs) accessible through the Census website and used by Census Help Line operators to respond to callers’ questions.

This supplemental PIA will be publicly available on the Statistics Canada website as an addendum to the Generic PIA. In addition, Statistics Canada has prepared FAQs and speaking points to assist its internal staff to deal with inquiries from the public regarding the Test.

Other factors:

This section identifies other privacy considerations related to the 2024 Census Test.

Use of administrative data for collective dwellings

Statistics Canada attempts to collect the same information from residents in collective dwellings that is collected from other Canadians and has used various collection approaches in the past. The 2024 Census Test will repeat the methodology used for the 2021 Census whereby the administrators of collective dwellings are required to answer a series of questions about the facility or establishment and complete the Census Test questionnaire for residents. For the 2024 Census Test, administrators will primarily use the electronic questionnaire, whereas data for collective dwellings without administrators will be collected by field enumerators. If the administrator is unable to complete the Census Test questionnaire online, a Census Test employee will follow up by phone or in-person. Residents will also be able to self respond if they wish using the electronic questionnaire.

Test of “informed replacement” letter

As part of the 2024 Census Test, with the goal of reducing response burden, a sample of approximately 8,700 respondents will be provided with an “informed replacement” letter with an offer by Statistics Canada to complete their questionnaire using data already provided to the government. Respondents can still choose to complete their questionnaire themselves if they prefer. The letter states that “Statistics Canada will automatically use information your household has already provided to other government sources about the number of residents, their age, and sex at birth or gender”, and includes a link to the Census website where respondents are provided with an exhaustive list of the administrative data that would be used should the dwelling choose this option. Qualitative testing for the content of this letter demonstrated that the sources of administrative data should be listed on the Census website rather than in the text of the letter itself. The best way to communicate the administrative data sources will continue to be tested. The results will be used to help evaluate the social acceptability and effectiveness of this approach for future censuses (post 2026 cycle).

Cellular phone text message and landline voice broadcast reminders

In 2021, as a final reminder before starting field follow-up activities with non-respondents, the census used text messaging, voice broadcast, and email to remind Canadians living in mailout areas (where mail delivery to a civic style address is possible) to complete their Census. Non-responding dwellings with cellular phone numbers received a single text message reminder. Non-responding dwellings associated only with a landline phone number received a voice broadcast reminder. The census also tested the use of email addresses collected during the 2016 Census (updated by the Labour Force Survey and/or the 2019 Census Test when possible) to send a reminder to non-responding dwellings for which no phone number was listed on the census file.^Footnote4

Results from the 2021 Census demonstrated that there was a clear increase in online responses for non-responding dwellings that received the text message or the voice broadcast reminder. On the other hand, email reminders were not effective to increase self response in the 2021 Census. Based on these results, the census explored opportunities to expand the use of text and voice broadcast messages. To determine email reminders’ full potential, additional testing is required but is being postponed indeterminately due to current operational limitations (maximum number of email reminders that can be sent per day is insufficient). 

Consultations with 2021 Census respondents and non-respondents revealed that some mailout households receive their census invitation letter late because they do not check their community mailboxes regularly. An early text reminder could incite more Canadians to check their mailboxes, complete their census questionnaire earlier, and avoid receiving additional reminders. During January 2023 qualitative testing, participants were asked about their initial reaction to such a text message.  Most of the reactions were quite neutral, while some suggested that it would prompt them to check their mailboxes and felt that it added significance to the letter. Therefore, the 2024 Census Test will attempt to measure the impact of a text message on self-response. On May 16 and 17, 2024, all non-responding dwellings in-scope for the test (approximately 57,000 dwellings) will receive a text message. This timing has been chosen to encourage households to retrieve their census invitation letter from their mailbox close to Census Test reference day (May 14) while accounting for operational capacity. Dwellings selected for the test that have not responded around the end of May could also receive a text reminder on June 3, 2024 (approximately 36,000 dwellings). Additionally, the test will be used to measure the impact of a second text message on self-response. Note that some non-responding dwellings will only receive the June 3rd text reminder because of the design of the test and that approximately 20,000 dwellings associated with a landline phone number will receive a voice broadcast message.

Another area where the Census is considering expanding the use of text message and voice broadcast is outside the mailout areas. Non-responding dwellings located outside mailout areas have their invitation letters delivered by census staff. The only contact with these dwellings prior to the field follow-up activities is a postcard sent to all dwellings to thank respondents and ask non-respondents to complete their census questionnaire as soon as possible. The postcard has not demonstrated to have a significant impact on self-response. The 2024 Census Test will attempt to measure the impact on self-response of sending a text or a voice broadcast reminder to non-responding dwellings. On May 26, 2024, non-responding dwellings outside the mailout areas in scope for the test (approximately 2,000 dwellings) will receive a text or a voice broadcast reminder. The type of reminder received by a dwelling will depend on the type of telephone number present on the census file.

Statistics Canada’s website will provide answers to questions related to the collection of phone numbers and the use of text message and voice broadcast campaigns. Note that on the third mail reminder (only in mailout areas), non-respondents will be informed that they can subsequently be contacted by Statistics Canada by phone, text message, email, or in person.

Statistics Act employee recruitment

Prospective candidates for Census Field Operations are required to complete an application through the secure, web-based Census Online Recruitment Application. The personal information associated with the application is stored in the Collection Management Portal (Statistics Canada's secure corporate tool for field staff), which is addressed in Statistics Canada's Generic PIA. This information includes name, address, contact information, citizenship status, experience, abilities, and references. This information is used by recruiters to screen, test, and interview candidates, as well as to initiate the security clearance process.

In compliance with the Policy on Government Security, all personnel hired for the census must be granted Reliability Status. This process includes verifying personal data, employment history and references. It also involves declaring any criminal convictions for which a pardon has not been granted. A criminal record name check is completed via the Royal Canadian Mounted Police database, and a credit check is conducted by a licensed credit reporting agency. All security checks are performed by personnel security officers on behalf of Statistics Canada. Approximately 600 candidates are expected to undergo security screening for the 2024 Census Test.

Census chatbot:

Statistics Canada will use a new chatbot for the 2026 Census to provide Canadians with timely, accurate, and automatic responses to frequently asked questions and a pathway to a live agent, if needed. The chatbot does not collect personal information and only uses responses that have been written and reviewed by subject-matter experts from Statistics Canada to ensure accuracy. Statistics Canada is developing a separate Privacy Impact Assessment (PIA) for the Census Chatbot which will be tested during the 2024 Census Test. Statistics Canada is committed to using artificial intelligence in a responsible and ethical manner and operates in accordance with governing instruments and frameworks that guide its responsible use including all Government of Canada directives on the responsible use of artificial intelligence.

New tool to retrieve secure access codes:

Respondents are required to provide their secure access code^Footnote5 to complete the Census Test on-line questionnaire. Statistics Canada will test the use of a new on-line tool that enables respondents to retrieve their secure access code using their address. Previously, respondents could only retrieve their code by calling the Census Help Line. The purpose of this initiative is to reduce burden for respondents by offering an alternative to calling the Census Help Line.

Conclusion:

This assessment concludes that, with the existing Statistics Canada safeguards, any remaining risks are such that Statistics Canada is prepared to accept and manage the risk.

Evaluation Report

February 2024

How the report is structured

• The report in short
• Acronyms and abbreviations
• What is covered
• What we learned
  • 1. Performance—achievement of expected outcomes
  • 2. Performance—planning process
  • 3. Performance—communication strategy and activities
• How to improve the program
• Management response and action plan
• Appendix A – Interview quantification scale

The report in short

Statistics Canada has a mandate and legal obligation under the Statistics Act to conduct the Census of Agriculture (CEAG) every five years. The CEAG provides a comprehensive profile of the physical, economic, social and environmental aspects of Canada's agriculture industry. It is the only data source that consistently provides high-quality detailed statistical information on agriculture for small geographic areas and collects a wide range of data at the national, provincial and subprovincial levels.

The CEAG provides both a snapshot in time of the agriculture industry and changes over time that are essential to inform public and private decision making. Its expected outcomes are to provide high-quality data and information that is accessible, available and relevant to all data users.^{Footnote 1} In addition to achieving the CEAG outcomes, the 2021 CEAG was expected to

• decrease response burden on farm operators
• increase integration with Statistics Canada's centralized infrastructure
• increase harmonization between the methods, concepts and processes of the CEAG and other economic statistics programs
• publish more data without compromising the confidentiality of respondents
• align the business model with Statistics Canada's modernization objectives.^{Footnote 2}

The evaluation was conducted in accordance with the Treasury Board Policy on Results and Statistics Canada's Risk-Based Audit and Evaluation Plan (2023/2024 to 2027/2028). The objective of the evaluation is to provide credible and neutral information on the ongoing performance of the CEAG and to identify potential areas to consider for the CEAG's next cycle.

The evaluation methodology consisted of a document review and interviews. Interviews were conducted with Statistics Canada staff, as well as with data users external to Statistics Canada. The findings outlined in this report are based on the triangulation of these data collection methods.

Key findings and recommendations

The CEAG program achieved its expected outcomes of providing high-quality data and information that is accessible, available and relevant to all data users with the timely delivery of the 2021 CEAG.

Compared with 2016, the 2021 CEAG both sustained and improved on various factors that contribute to producing high-quality data and to their relevance to users, their availability and their accessibility. The 2021 CEAG made progress on its objectives to improve operational efficiencies through integration with Statistics Canada infrastructure, increase harmonization with other economic statistics programs and reduce response burden. As with transformational changes, full potential benefits will be realized moving forward as the processes and activities become fully integrated and more familiar to staff.

In general, the main overarching planning processes for the 2021 CEAG were similar to the 2016 cycle, with the addition of several significant methodological and operational changes: the change in the farm definition, the migration to the Integrated Business Statistics Program, the continued development of the electronic questionnaire and adaptations made because of the pandemic. The risk management process contributed to achieving expected results, as well as mitigating the risks that are specific to the changes made in the 2021 cycle. The significant number of changes for the 2021 CEAG made planning more challenging because of some unpredictable implications from the changes, and staff turnover added additional planning constraints. These large-scale changes implemented for the 2021 CEAG have also fundamentally altered many processes, establishing a new baseline moving forward.

Overall, the 2021 CEAG communication strategy and activities were effective in maintaining, and in some cases improving, awareness of CEAG products. New and improved tools increased awareness and facilitated users' understanding of how to interpret CEAG data. Some areas noted for consideration for subsequent cycles of the CEAG are continuing to amplify the CEAG through various communication and engagement channels; empowering users to access, use and interpret the data; and providing clearer information and guidance on changes.

In light of these findings, the following recommendations are proposed:

Recommendation 1

The Assistant Chief Statistician (ACS), Economic Statistics (Field 5), should

1. ensure that the process in place to regularly review resources includes revisiting the balance between upcoming and planned deliverables (including those for communication and engagement) and the corresponding challenges versus available time and resources
2. in addition to the regular review process, given that the 2026 CEAG is not expected to include any new large-scale changes, verify that the planned levels and allocation of resources are correctly aligned with the new baseline resulting from the changes made for the 2021 CEAG; this will help ensure that future core activities are resourced appropriately.

Recommendation 2

The ACS, Economic Statistics (Field 5), should ensure that effective training, retention and succession plans are developed, reviewed regularly and aligned with future needs of the CEAG program.

Recommendation 3

The ACS, Economic Statistics (Field 5), should ensure that the CEAG has a communication and engagement plan in place that includes

1. varied communication approaches to broaden reach and clearly communicate the CEAG's intentions and plans, including the corresponding benefits, to increase stakeholders' understanding and acceptance
2. adequate approaches to improve data users' awareness of and engagement with CEAG data, to further support CEAG tool usage and data interpretation, including working with users with less statistical capacity.

Acronyms and abbreviations

ACS

Assistant Chief Statistician

ASP

Agriculture Statistics Program

ATDP

Agricultural Taxation Data Program

CAMT

Census of Agriculture Management Team

CASC

Census of Agriculture Steering Committee

CEAG

Census of Agriculture

CePop

Census of Population

DPMO

Departmental Project Management Office

EQ

Electronic questionnaire

FPT

Federal-Provincial-Territorial

IBSP

Integrated Business Statistics Program

NFD

New farm definition

OMG

Operations Management Group

PMT

Project Management Team

SMC

Strategic Management Committee

What is covered

The evaluation was conducted in accordance with the Treasury Board Policy on Results and Statistics Canada's Risk-Based Audit and Evaluation Plan (2023/2024 to 2027/2028). The objective of the evaluation is to provide credible and neutral information on the ongoing performance of the Census of Agriculture (CEAG) and to identify potential areas to consider for the CEAG's next cycle.

The Census of Agriculture

Mandate, outcomes and objectives

Statistics Canada has a mandate and legal obligation under the Statistics Act to conduct the CEAG every five years. The CEAG provides a comprehensive profile of the physical, economic, social and environmental aspects of Canada's agriculture industry. It is the only data source that consistently provides high-quality detailed statistical information on agriculture for small geographic areas and collects a wide range of data at the national, provincial and subprovincial levels. The types of data collected include the number of farms and farm operators, farm area and size, land management practices, business operating arrangements, farm operating revenues and expenses, and farm capital. The CEAG provides both a snapshot in time of the agriculture industry and changes over time that are essential to inform public and private decision making.

The expected outcomes for the CEAG are to provide high-quality data and information that is accessible, available and relevant to all data users.^{Footnote 3} To achieve its outcomes, the CEAG is expected to

• provide statistical information and analysis about Canada's economic and social structure to develop, evaluate and improve public policies and private decision making
• promote sound statistical standards and practices to achieve greater efficiency in data collection
• provide critical information to manage federal and provincial governments' expenditures in support of the agriculture sector
• benchmark agriculture information to realign annual and subannual survey estimates and economic data to ensure accuracy and coherence
• provide agriculture information for small geographic areas based on complete enumeration to inform program and policy monitoring
• provide an accurate frame of all farms and farm types to inform the agriculture survey program
• provide measurement of rare or emerging commodities to inform disease control or trade issues.

In addition to achieving the CEAG outcomes, the 2021 CEAG was expected to

• decrease response burden on farm operators
• increase integration with Statistics Canada's centralized infrastructure
• increase harmonization between the methods, concepts and processes of the CEAG and other economic statistics programs
• publish more data without compromising the confidentiality of respondents
• align the business model with Statistics Canada's modernization objectives.^{Footnote 4}

Census cycle

The CEAG is conducted in May to align with the Census of Population (CePop) and take advantage of cost savings achieved by leveraging the CePop's resources and infrastructure. Every CEAG census cycle has a six-year timespan, during which Statistics Canada conducts several activities. Figure 1 below highlights the six broad components that make up the CEAG's six-year cycle. While the activities are presented in a linear progression, many activities across projects happen simultaneously.

Description - Figure 1: The six components and associated key activities of the Census of Agriculture cycle

Figure 1 depicts the six broad components that make up the CEAG's six-year cycle.

1. Content determination: 
   • Consult with user groups across the country to inform Census of Agriculture (CEAG) questions (e.g., identify data gaps and needs of data users).
   • Test content changes, including the proposed questions, electronic questionnaire, edit and imputation processes, and validation tools.
   • Seek Cabinet approval and publish CEAG content in The Gazette.
2. Communications:
   • Develop and launch a campaign to raise awareness of the CEAG, promote self-response through online collection and increase self-response rates.
   • Promote messaging through various communication channels, including outreach, public and media relations, social media, respondent relations, and census website management.
3. Collection:
   • Develop the census frame using Statistics Canada's Business Register and tax remittances.
   • Collect data online or (on demand) on paper (questionnaire forms) from May 3 to August 29, 2021.
   • Distribute email, letter and voice reminders.
   • Follow up with non-respondents to complete the questionnaire.
4. Processing:
   • Conduct initial processing: geo edit to assign province or territory, census agricultural region, division and subdivision; code open-ended responses; remove duplicate questionnaires; and link farm operators with people in the Census of Population database.
   • Import data into the Integrated Business Statistics Program and integrate them with other data sources, address data errors through edit and imputation, and generate estimates through summation.
5. Data quality evaluation:
   • Validate data by finding and correcting significant errors at the provincial and subprovincial levels, then at the farm level, and finally conduct a macro-level review to assess final estimates.
   • Develop and present certification reports summarizing the data validation process and recommendations for publication.
6. Dissemination:
   • Develop and release informational products, including farm and farm operator data, provincial and territorial profile articles, analytical articles, agriculture-population linkage data, the Guide to the Census of Agriculture and reference and thematic maps, a mapping tool, and community profiles.
   • Generate and distribute custom data products requested by data users.

Governance structure

The CEAG is overseen by the Strategic Management Committee (SMC), a senior governance body chaired by the Chief Statistician and composed of assistant chief statisticians (ACSs).

The Census of Agriculture Steering Committee (CASC) reports to the SMC and directs the Census of Agriculture Management Team (CAMT). The CASC is chaired by the Director General, Agriculture, Energy and Environment Statistics Branch (the business sponsor), and consists of the CEAG manager and directors from various Statistics Canada divisions and stakeholders from Agriculture and Agri-Food Canada. Beyond providing direction to the CAMT, the CASC is responsible for monitoring project progress; approving recommendations for changes to scope, schedule and cost; and providing advice and assisting with the resolution of issues between stakeholder groups.

The CAMT is responsible for the ongoing decision making and management of the CEAG, including risk management and mitigation planning. The CAMT is composed of the Census Manager and subproject managers (e.g., content determination, collection, processing, validation, certification and dissemination). The CAMT meets weekly and as necessary, and subproject teams meet biweekly and report progress and issues to the CAMT.

External consulting bodies engaged in the CEAG include the Advisory Committee on Agriculture Statistics and the Federal-Provincial-Territorial (FPT) Committee on Agriculture Statistics. These bodies provide feedback on emerging issues and informational needs within the agricultural sector.

Internal management committees also help direct the CEAG. The Economic Statistics Field Planning Board reviews new project proposals, is responsible for making final decisions for reviews of all projects in the field, and monitors project execution and the allocation of information technology resources. Finally, initiatives related to the Integrated Business Statistics Program (IBSP) fall under the IBSP Project Management Team (PMT).

The evaluation

The scope of the evaluation covered ongoing program performance, including changes made between the 2016 and 2021 census cycles, as well as performance from the lens of planning and communications. The scope was established in collaboration with the office of primary interest. The evaluation was conducted from July to October 2023.

The three evaluation questions that were identified are the following:

1. To what extent has the 2021 CEAG achieved its expected outcomes?^{Footnote 5}
2. To what extent did the 2021 CEAG's planning process effectively support the achievement of expected outcomes?
3. To what extent have the 2021 CEAG's communication strategy and activities effectively supported the achievement of expected outcomes?

The data collection methods outlined in Figure 2 were used.The findings outlined in this report are based on the triangulation of these data collection methods.

Description - Figure 2. Data collection methods

Figure 2 depicts the three data collection methods used for the evaluation: external interviews, internal interviews, and document review.

The external interviews included semi-structured interviews data users from provincial government organizations, academia and agriculture organizations. There were 20 interviews conducted with 26 people.

The internal interviews included semi-structured interviews with Census of Agriculture representatives, as well as partners within Statistics Canada. There were 8 interviews conducted with 8 people.

The document review included a review of Statistics Canada's files, documents, and web trends information.

Two main limitations were identified, and mitigation strategies were employed, as outlined in Table 1.

Table 1. Limitations and mitigation strategies

Limitations	Mitigation strategies
The perspectives gathered through external interviews may not be fully representative.	External interviewees were selected using specific criteria to maximize strategic reach for the interviews. Multiple recruitment strategies were used. Evaluators were able to find consistent overall patterns.
There were a few internal interviewees who were unavailable to participate, and therefore, their perspectives are not represented in the evaluation.	To address this gap, documents—particularly the task evaluation reports—were used to provide additional evidence, and alternate interviews were conducted.

What we learned

1. Performance—achievement of expected outcomes

Summary

The CEAG program achieved its expected outcomes by providing high-quality data and information that is accessible, available and relevant to all data users with the timely delivery of the 2021 CEAG. Compared with 2016, the 2021 CEAG both sustained and improved on various factors that contribute to the expected outcomes. It made progress on its objectives to improve operational efficiencies through integrating with Statistics Canada infrastructure, increasing harmonization with other economic statistics programs and reducing response burden. As with transformational changes, full potential benefits will be realized moving forward as the processes and activities become fully integrated and more familiar to staff.

The 2021 CEAG was expected to provide high-quality statistical information that is relevant to its users, available and accessible. In addition, the 2021 CEAG intended to reduce respondent burden, improve operational efficiencies^{Footnote 6} through integration with Statistics Canada infrastructure and increase harmonization with other economic statistics programs.

Compared with 2016, the 2021 CEAG both sustained and improved on various factors that contribute to producing high-quality data and to their relevance to users, their availability and their accessibility.

Improvements to data quality resulted from the promotion and uptake of the online electronic questionnaire (EQ), achieving 82% uptake and exceeding the target by 12%. The uptake of the EQ contributes to data quality by preventing certain respondent errors. For example, when unlikely or invalid data were entered, the EQ initiated a prompt for the respondent to check their response, and the EQ automatically filled in certain fields based on information from the Business Register. Other factors contributing to data quality include the response rate and coverage errors. While the 2021 response rate was lower than in 2016, and the 2021 farm count undercoverage rate was higher than in 2016, neither of these factors is directly comparable between the two CEAG cycles because of new methodologies used in 2021.^{Footnote 7}

Overall, the 2021 CEAG data coverage and relevance were similar to 2016, with some improvements resulting from the content consultations with census stakeholders (federal departments, provincial ministries, agriculture associations, etc.). The addition and refinement of questions about sustainability practices and technology adoption in the agriculture sector improved the breadth and depth of data coverage, as well as relevance to data users.

The timeliness of releases for some of the 2021 CEAG products was improved, compared with the 2016 CEAG cycle. New products, including reference maps, a mapping tool and the CEAG Guide, were released one to three months ahead of the farm and farm operator data. The timing of the release for the first key CEAG data output, the farm and farm operator data, was similar to that of the 2016 cycle—one year following Census Day, which is the day that the census is rolled out. The following CEAG outputs were also released earlier, compared with the 2016 cycle:

• the agriculture-population dataset that links CEAG and CePop data was released three months earlier
• the reference maps, which provide the geographical boundaries, codes and names for all geographic areas appearing in the data tables, were released approximately three months earlier
• custom data orders requested by end users were reported to be released faster in 2021.

While there were a few end users who reported that timelier CEAG data could better meet their needs, or questioned why the transition to an EQ did not result in faster data releases, the general consensus was that there is always a preference for high-quality data over earlier data, and the timelines for CEAG data were appropriate.

The availability of data was improved by publishing (for the first time) quality indicators in the farm and farm operator data to help data users determine to what extent they may want to use certain data points to inform decisions, policies or programs. Data availability was further improved through the introduction of a new disclosure methodology^{Footnote 8} that decreased data suppression—a noted improvement for a few key external interviewees. Improvements to data access included making microdata available in research data centres for researchers with research proposals to access directly. Additional accessibility improvements are discussed in the "Performance—communication strategy and activities" section.

The maintenance of and improvement to elements of data quality were facilitated by the promotion of an EQ, the content consultation process and the experience level of data users.

As already stated, data quality improvements were facilitated by the promotion and uptake of the EQ. Data coverage and relevance improved too, because of the content consultations that informed questions about sustainability practices and technology adoption. In addition, consultations following the 2016 CEAG cycle with key stakeholders (e.g., federal and provincial or territorial governments, Agriculture and Agri-Food Canada) facilitated improvements in data access and use through the addition of new data visualization and mapping tools. Custom datasets were reported as accessible and timely for long-time data users, given that the requested data were often similar from cycle to cycle and the process was well understood. For less experienced CEAG users (and some smaller producer organizations), the process (e.g., where to direct a request, how to frame the data request) and costs associated with requesting custom data were viewed as barriers.

Operational and methodological changes tempered some of the improvements to data quality and coverage, while other improvements to data quality and availability were facilitated by operational changes.

The methodological change to calculating the CEAG coverage errors generated higher rates of undercoverage; however, the 2021 estimates are considered to be more accurate as a result of the improved methodology. Although planned additional improvements to include detailed revenue data from the Agriculture Taxation Data Program (ATDP) for publication did not go forward because of concerns related to accuracy, the ATDP data replacement improved overall data coverage and quality because it avoided missing values from refusal or false information. Furthermore, the migration to the IBSP enabled the publication of data quality indicators and, as a result, improved data availability.

The 2021 CEAG made progress on its other objectives to improve operational efficiencies through integration with Statistics Canada infrastructure, to increase harmonization with other economic statistics programs and to reduce response burden. Full potential benefits will be realized moving forward as the processes and activities become fully integrated and more familiar to staff.

Operational efficiency gains were realized through the migration of the 2021 CEAG to the IBSP. The migration to the IBSP supports operational efficiencies through data integration from other programs; earlier data validation because of rolling estimates; and the use of an existing corporate tool, expertise and experience, as opposed to a stand-alone processing system. From the perspective of internal interviewees, the new processes, approaches and sustained effort required to address the challenges^{Footnote 9} related to the IBSP migration limited the gains for the 2021 CEAG cycle, but the lessons learned from the migration are expected to yield more efficiencies for subsequent CEAG cycles.

The introduction of a new farm definition (NFD) for the 2021 CEAG increased harmonization with other economic statistics programs.^{Footnote 10} This NFD facilitated the flow of data from other economic programs, such as the ATDP, into the CEAG and supports the comparison and interpretation of data across all agriculture statistics programs that share the same definition. Furthermore, the NFD supports harmonization with tax data and the Statistics Canada Business Register, used to develop the census frame.

The 2021 CEAG decreased response burden through the use of administrative data replacement, filter questions, an optimized online EQ and the use of tax data to inform the sample frame. Data replacement decreased response burden by using existing data to replace questions about revenues and expenses, sex, age, and operating arrangement. Filter questions reduced response burden, since only questions that were relevant to each respondent's operations were asked. Finally, the new census frame decreased response burden for out-of-scope operations (e.g., hobby farms) that no longer receive the CEAG and for CePop respondents who no longer had to answer a question about agriculture revenue. While data replacement theoretically reduced the time to complete the 2021 CEAG by 10% (or 3.3 minutes), the actual time to complete the CEAG decreased by 30 seconds because of additional qualifying questions designed to ensure that respondents were in scope.

The greater integration with Statistics Canada infrastructure and the harmonization with economic statistics programs were both facilitated and hindered to varying degrees by similar factors: the experience level of team members, project management and the COVID-19 pandemic.

Internal interviewees and the documentation review detailed facilitators and hindrances to the tasks involved with, or resulting from, IBSP integration and harmonization with economic statistics programs, such as the following:

• Some activities involved with the migration to the IBSP (e.g., taxes and modelled births added into the population and imputed into the IBSP) were facilitated by strong collaboration and consultation, while other activities could have benefited from more consultation and collaboration with other Agriculture Statistics Program (ASP) teams (e.g., validation of revenues and expenses integrated from the ATDP).
• The presence of some experienced CEAG team members and ASP staff facilitated the necessary process adaptations or workarounds required to accommodate the migration to the IBSP; however, the data validation activities caused several challenges for the validation team, which did not have prior experience with the new iterative process and the interdependencies between detailed, subtotal and total values in the IBSP.
• The larger staff recruitment pool (pan-Canadian) resulting from the remote work policies related to COVID-19 facilitated some production-related activities; however, remote training challenges early on in the pandemic hindered training.
• Some activities benefited from well-defined project schedules, roles and responsibilities (e.g., the NFD to harmonize with economic statistics programs and integrate ATDP data within the IBSP), while for others, there was a feeling that more planning and management would have been beneficial (e.g., last-minute organizational plan for validation of revenue and expense data).

The commitment of staff to the CEAG helped teams address the various challenges associated with the 2021 CEAG changes, but tight timelines did not permit sufficient opportunity to research, develop, test, analyze and revise some activities as thoroughly as staff would have liked (e.g., backcasting, data validation, NFD and sampling frame), leading to unexpected challenges that required additional work, including overtime.

The 2021 CEAG methodological and operational changes impacted the CEAG work environment, and changes to decrease response burden affected data processing.

The number and complexity of operational and methodological changes implemented in 2021 contributed to a heightened level of stress for many team members.

The transition to the new environment required several workarounds that were largely understood and developed by staff who had extensive experience with the CEAG program—many of whom have moved on, or (as of the fall of 2023) are expected to move on, from the program. This gap in institutional knowledge creates challenges for the 2026 Census cycle, which will have fewer team members with experience in the full six-year census cycle.

In addition, the EQ filter questions decrease response burden by asking a series of yes or no questions to determine what questionnaire sections apply to each respondent. However, the number of unexpected answers to these questions was higher than anticipated, resulting in more corrections and higher imputation rates. For example, it was found that a high number of respondents indicated that they had no machinery. This then required the data validation team to identify records that likely did have machinery and manually fix the answer before sending the records to imputation.

The replacement of some CEAG respondent data with those from other administrative sources (part of AgZero^{Footnote 11}) is advancing one of the five pillars of the agency's modernization agenda. Some external interviewees relayed their concern that transitioning to administrative or survey data may limit data at smaller geographic levels, where administrative data may not be available.

2. Performance—planning process

Summary

In general, the main overarching planning processes for the 2021 CEAG were similar to the 2016 cycle, with the addition of several significant methodological and operational changes: the change in the farm definition, the migration to the IBSP, the continued development of the EQ and adaptations made because of the pandemic. The risk management process contributed to achieving expected results, as well as mitigating the risks that are specific to the changes made in the 2021 cycle. The significant number of changes for the 2021 CEAG made planning more challenging, and staff turnover added additional constraints.

The main steps of CEAG planning were the same as in the previous cycle. The addition of several significant methodological and operational changes affected planning: the change in the farm definition, the migration to the IBSP, the continued development of the EQ and adaptations made because of the pandemic.

The overall governance for the CEAG planning process for the 2021 cycle was similar to the 2016 cycle. However, the Operations Management Group (OMG) was introduced in March 2021 to manage the CEAG production. The OMG core members^{Footnote 12} met daily from April 29, 2021, to March 4, 2022, to identify and address CEAG production-related issues. As well, the inclusion of the IBSP PMT provided oversight for all programs migrating to the IBSP framework. Furthermore, the following revisions to the organizational structure occurred in the 2021 cycle:

• the CEAG dissemination unit expanded to do more divisional work
• CEAG processing responsibilities shifted to the Enterprise Statistics Division, which is responsible for the IBSP
• client services (i.e., custom data) transitioned to Statistics Canada's regional data service centres.

The main overarching planning processes for the 2021 CEAG were similar to the 2016 cycle. The content determination project included its usual national consultation process to inform the instrument design with data users in 2017; questions were tested, and the CEAG team continued to explore steps that could be partially or entirely replaced by administrative data. The communication project established a plan to increase awareness and promote participation in the CEAG, while leveraging the communication efforts of the CePop. The collection, data quality and dissemination projects all developed strategies and plans to ensure that their activities were successfully executed. Planning oversight by the CEAG management teams, including the CAMT and unit heads, continued through regular meetings over the course of the CEAG cycle.

In addition, the 2021 planning process had to accommodate several large-scale changes, some of which were related to the agency's modernization and innovation agenda. Specifically, the 2021 CEAG had to plan for the following changes:

• implementing the NFD
• increasing data integration
• executing a new data processing environment
• pursuing further development of the EQ to reduce response burden
• realizing a new disclosure avoidance method.

Planning started prior to the Treasury Board submission for the 2021 CEAG cycle, specifically for activities involving the NFD and migration to the IBSP, and new methodologies were developed for CEAG data validation to accommodate the IBSP circular processing model. However, a few internal interviewees reported that, as a result of the extent and complexity of the changes, they could have benefited from more time to plan and implement them.

The 2021 CEAG risk management process contributed to the achievement of expected results by identifying and planning for risks that are standard to most CEAG cycles, as well as risks specific to the changes planned for the 2021 cycle.

Throughout the CEAG cycle, a risk register was managed by the CAMT, and risks were reported on at the monthly CASC meeting. Risks unique to the 2021 CEAG were primarily related to harmonization with other economic statistics programs, namely the new census frame; the NFD; and migration into the IBSP. These risks, along with their predicted impact and associated mitigation plans, are further described below.

• New census frame: Risks identified with the new definition included potential impacts on data quality. The exclusion of in-scope farms from the census frame was a potential data-quality risk resulting from the NFD and dependency on the Business Register to inform the frame. To mitigate this risk, the new frame was tested against the 2016 Census frame and the Agriculture Frame Update Survey to ensure alignment, and modelled births were included in the CEAG to account for new farms since the 2020 tax data.
• NFD: The effect of the NFD on the comparability of 2021 CEAG data with 2016 CEAG data was a data relevance risk from end users' perspective. To mitigate this risk, a backcasting exercise was undertaken to better understand how to interpret the 2021 CEAG data, compared with 2016.
• Migration to IBSP: One of the key risks reported by a few internal interviewees regarding the IBSP migration was related to the IBSP's ability to process the volume of data involved in the census, possibly disrupting the timelines. To mitigate this risk, the CEAG was tested repeatedly in the IBSP environment prior to production, and CEAG teams consulted with other teams that had undergone the transition to the IBSP, as well as the IBSP managing team.

Other risks classified as being low or moderate and, for the most part, deemed manageable included low response rates, challenges in securing enough skilled human resources at the right time and natural disasters.

In addition to the planned risks, the 2021 CEAG also needed to account for unanticipated risks, namely those pertaining to the COVID-19 pandemic. While the program has a business continuity plan, it did not account for the scale and ramifications of the pandemic. The planning for a variety of 2021 CEAG activities had to pivot because of COVID-19. For example,

• the communication and engagement plan transitioned from including in-person engagement (e.g., attendance at agricultural shows) to being completely online
• the data replacement plan was revised to ensure that industry data impacted by the pandemic were excluded from the replacement strategy
• the training plan was revised to be online and accommodate a pan-Canadian workforce working from home (as opposed to onsite in the National Capital Region).

Facilitators in CEAG planning included the matrix management approach and regular planning meetings. The significant changes for the 2021 CEAG made planning more challenging, given the complexity and magnitude of the changes. Staff turnover was an additional challenge.

The CEAG planning process used a matrix management approach to help ensure the interdependencies across divisions and activities outside the Census Manager's reporting structure were kept on track. A few internal interviewees reported that the regular planning meetings between the unit heads for collection, processing and dissemination, as well as cross-unit planning meetings between the CEAG and CePop, also facilitated the planning process. As already stated, the planning around the IBSP migration was facilitated by consultations with other teams that had undergone the transition. Because of the unique and complex nature of the CEAG, there were unexpected challenges (e.g., dealing with the infrastructure of the processing environment, CEAG descriptive mnemonics versus IBSP naming conventions) that the CEAG team had to address that were not experienced by other teams.

The substantial changes introduced in the 2021 CEAG cycle (e.g., IBSP migration and NFD) were reported to be the most significant planning challenge. A few internal interviewees reported that they felt that the complexity of the changes was not adequately planned for, despite factoring in the potential challenges in the planning process. Other challenges included staff turnover, especially at the critical levels (e.g., key management personnel and subject-matter experts), that left gaps in knowledge and hindered planning.

For the amount and level of change implemented for the 2021 CEAG, resources were viewed as being very tight for the core CEAG team.

Significant effort went into planning for the 2021 cycle. In addition, efforts to identify lessons learned from the various tasks for the 2021 CEAG cycle provided detailed feedback to inform planning for future cycles. Some internal interviewees reported that there was insufficient time to fully plan for the unprecedented scale of changes. It was suggested by some interviewees that incorporating the significant changes more gradually would have alleviated some pressure; however, it was noted during a subsequent discussion that gradual implementation across more than one cycle was not possible for the 2021 CEAG because of the interrelated nature of the changes.

The 2021 CEAG also experienced human resource challenges throughout the development, production and dissemination phases, especially given the activities planned. Key departures during the development and production cycle increased the level of stress for the team and the risk for program delivery. For example, during interviews, it was noted that when a CEAG team member left unexpectedly, their tasks were held up because there was no one available to immediately fill in for the missing resource.

The recruitment of junior and senior validators was delayed because of challenges in securing a sufficient number of qualified validators for the 2021 CEAG.^{Footnote 13} As a result, the recruitment efforts had to use initiatives that were not pre-planned or pre-approved (e.g., exemption from language policy for senior validators). According to the lessons learned document, the planning and resourcing of the dissemination activity cycle occurred too late, making it challenging to take full advantage of the resources.

3. Performance—communication strategy and activities

Summary

Overall, the 2021 CEAG communication strategy and activities were effective in maintaining, and in some cases improving, awareness of CEAG products. New and improved tools increased awareness and facilitated users' understanding of how to interpret CEAG data. Some areas noted for consideration for subsequent cycles of the CEAG are continuing to amplify the CEAG through various communication and engagement channels; empowering users to access, use and interpret the data; and providing clearer information and guidance on changes.

The 2021 CEAG communication and dissemination strategy supported the achievement of high-quality data that are accessible, available and relevant.

CEAG communications prior to Census Day and throughout the collection period contributed to data quality and efficiency by promoting participation through messaging about the benefits of completing the questionnaire and methods used to reduce response burden (e.g., data replacement). Diversifying the communication channels during collection and dissemination, including social media campaigns, the census website and the FPT Committee on Agriculture Statistics, helped to reach the various stakeholder groups. External interviewees who were members of the committee reported being satisfied with the regular CEAG communication updates at their quarterly meetings.

Communication and dissemination efforts also contributed to the accessibility and availability of data through the release of tools and guidance materials before the official data release in May 2022, a new online portal housing all CEAG-related content and publications, new interactive data visualization and mapping tools, and webinars highlighting CEAG products. The dissemination activities also supported data relevance through the publication of analytical reports focusing on cross-cutting themes and emerging sectors.

The COVID-19 pandemic led to the transition of all promotional activities to online engagement, but most external key interviewees did not perceive this transition to have limited promotion and awareness-raising activities, as more people transitioned online. However, a few external interviewees noted that additional methods to increase awareness of the CEAG and its products may be beneficial (e.g., farm shows, radio spots, promotional materials and direct emails).

The 2021 CEAG also used a combination of articles, infographics, and new interactive data visualization and mapping tools to support end users' engagement with the data. Microdata were also made available to researchers at the research data centres. Data users reported that making data available using a variety of methods, especially the infographics and visualization tools, improved data access and use.

The communication strategy and dissemination activities were effective in maintaining, and in some cases improving, awareness of CEAG products.

The majority of external interviewees reported that their awareness of CEAG products was similar to the previous census cycle, because they are often returning to the same products after every census cycle. However, some data users reported being made aware of the new data visualization and mapping tools, and the timeline of releases. Although web analytics data for the 2021 CEAG could not be compared with those from 2016 because of software changes within Statistics Canada, the uptake of the mapping and data visualization tools was relatively stable from month to month, indicating steady and ongoing use.

End users often reported The Daily as the main source of CEAG-related updates, releases and news, and the new CEAG portal as the primary access point to CEAG data. However, users also noted that they would use Google to search for relevant data, as opposed to going through the official Statistics Canada website. A recommendation to improve awareness of CEAG products was to email stakeholder groups a week or two prior to a product's release.

Overall, most data users reported being satisfied with the accessibility of CEAG data. The majority of data users accessed CEAG data directly through the new CEAG portal, which, for the most part, was considered user friendly. While the portal was revised based on early stakeholder consultation, a few data users described challenges finding the relevant data on the portal. In addition to the online portal, some external interviewees accessed CEAG data through custom data requests via email.

End users' understanding of how to use CEAG tools was largely maintained or improved as a result of the new user guide and webinars. Their understanding of how to interpret CEAG data was supported by data visualization tools, infographics and articles.

Most external interviewees reported that their understanding of how to use the CEAG tools remained the same, largely as a result of being long-time users of CEAG data. The publication of the user guide improved understanding of some of the changes to the 2021 CEAG and some of the new analytical tools that were introduced. Although the methodological changes were presented to various partners throughout the planning cycle (e.g., FPT Committee on Agriculture Statistics), a few external interviewees reported that the CEAG could do more to engage with data users early on to help them understand the methodological changes and how they will affect their analysis and interpretation of CEAG data.

Webinars also supported data users' understanding of how to use the CEAG tools. However, there are still opportunities to improve data users' understanding of how to use the tools (e.g., tables), particularly for those who are inexperienced with CEAG data. There was also a request for webinars teaching experienced data users how to use CEAG tools for more advanced analytical purposes.

Most data users reported that their understanding of how to interpret CEAG data remained similar to that for the 2016 CEAG. The continued publication of infographics and the new data visualization tools, along with the analytical articles, were all perceived to enhance understanding and use of CEAG data. A few external interviewees reported a desire for more analytical reports, including sector-specific reports that were available in past CEAG cycles.

Communication strategies and activities to consider for the next CEAG cycle include continuing to amplify and target CEAG promotion and data through social media and other means, for both engagement and dissemination activities, and empowering new and advanced end users to access, use and interpret CEAG data.

CEAG social media engagement was viewed positively by end users and internal interviewees. End users supported the continued use of social media to engage stakeholders and suggested that platform-specific campaigns be considered, depending on the target audience (e.g., X, formerly known as Twitter, was reported to be used more by producers).

Interviewees mentioned that the provision of webinars or other educational tools was useful and noted that tailoring them to levels of expertise (e.g., novice, advanced) would further support CEAG tool usage and interpretation. Additionally, some users commented that there is an opportunity to increase end users' awareness and understanding of what informational needs custom data requests can help fulfill, and the process involved in making a request.

Transitioning to publications that transcend all sectors may be undertaken in conjunction with more end-user education on how to find, use and interpret publicly available or customized data products.

The dissemination plan outlines the transition to a more sector-wide, horizontal approach to publications that are relevant to a wider audience. However, eliminating or limiting commodity-specific reports may need to be partnered with more support to end users to increase their knowledge and skills on how to find, use and interpret data, especially for smaller commodity sectors.

How to improve the program

Proper planning includes identifying, assessing and addressing risks and challenges, and helps to ensure that adequate time and resources are available for a program to deliver on its priorities and commitments to achieve its outcomes. About half of the interviewees noted that there were gaps in planning and resources for the 2021 CEAG, which led to significant pressures on staff. Given the challenges around staffing and the loss of experience and expertise resulting from departures, human resource planning will be critical for subsequent cycles. In addition, the large-scale changes implemented for the 2021 CEAG have fundamentally altered many processes, establishing a new baseline moving forward. The 2026 CEAG will provide an opportunity to verify that the planned resource levels and allocations match this new baseline.

Past initiatives to raise awareness of products and tools have been effective and well received; the CEAG should continue these efforts. Amplifying and targeting communication and engagement efforts will help increase understanding of CEAG products and tools, the intentions and goals of new methodologies and collection vehicles, and the corresponding benefits, thereby supporting further acceptance.

Recommendation 1

The ACS, Economic Statistics (Field 5), should

• ensure that the process in place to regularly review resources includes revisiting the balance between upcoming and planned deliverables (including those for communication and engagement) and the corresponding challenges versus available time and resources
• in addition to the regular review process, given that the 2026 CEAG is not expected to include any new large-scale changes, verify that the planned levels and allocation of resources are correctly aligned with the new baseline resulting from the changes made for the 2021 CEAG; this will help ensure that future core activities are resourced appropriately.

Recommendation 2

The ACS, Economic Statistics (Field 5), should ensure that effective training, retention and succession plans are developed, reviewed regularly and aligned with future needs of the CEAG program.

Recommendation 3

The ACS, Economic Statistics (Field 5), should ensure that the CEAG has a communication and engagement plan in place that includes

• varied communication approaches to broaden reach and clearly communicate the CEAG's intentions and plans, including the corresponding benefits, to increase stakeholders' understanding and acceptance
• adequate approaches to improve data users' awareness of and engagement with CEAG data, to further support CEAG tool usage and data interpretation, including working with users with less statistical capacity.

Management response and action plan

Recommendation 1

The ACS, Economic Statistics (Field 5), should

1. ensure that the process in place to regularly review resources includes revisiting the balance between upcoming and planned deliverables (including those for communication and engagement) and the corresponding challenges versus available time and resources
2. in addition to the regular review process, given that the 2026 CEAG is not expected to include any new large-scale changes, verify that the planned levels and allocation of resources are correctly aligned with the new baseline resulting from the changes made for the 2021 CEAG; this will help ensure that future core activities are resourced appropriately.

Management response

Management agrees with the recommendation.

As a large project, the CEAG currently follows the agency's Departmental Project Management Office (DPMO) planning and reporting procedures and will continue to do so. In addition, the team will conduct a review of its 2026 Census project planning processes, in consultation with the DPMO, to find improvements.

Given the substantial changes that took place during the 2021 Census, the team will ensure that its planning and reporting procedures reflect the new reality to confirm that the expected workload associated with deliverables assigned to a given team does not exceed its resource availability.

In addition to finalizing a 2026 project plan, the CEAG team will add integrated scheduling to its existing planning and also carefully review resources and deliverables using the DPMO dashboard as a key coordination vehicle. The integrated schedule will ensure that the deliverables for both subject-matter and service areas are aligned with the new 2026 baseline and that all activities are resourced appropriately.

The project plan will be regularly monitored using the DPMO monthly dashboard and monthly CASC approvals.

Deliverables and timelines

A project plan approved by the CASC will be available by October 2024, and an integrated schedule, also approved by the CASC, will be available by January 2026.

Recommendation 2

The ACS, Economic Statistics (Field 5), should ensure that effective training, retention and succession plans are developed, reviewed regularly and aligned with future needs of the CEAG program.

Management response

Management agrees with the recommendation.

The CEAG team will review its existing training and documentation activities and, in collaboration with the agency's Workforce and Workplace Branch, create a human resources plan, including training, retention and succession planning activities.

The plan will be reviewed regularly and include the establishment of partnerships with the agency's Workforce and Workplace Branch to ensure that staffing activities occur as required and seek exemptions from policies or directives that may impede human resources activities, e.g., determinate versus indeterminate status. Where possible, the CEAG team will leverage activities within the CePop program.

The plan will also focus on

• attracting and retaining high-performing employees
• developing more robust succession planning to stabilize transitions in times of staff turnover
• implementing practices to improve the integration of new employees, e.g., improving documentation.

Deliverables and timelines

A human resources plan approved by the CASC will be delivered by September 2025.

Recommendation 3

The ACS, Economic Statistics (Field 5), should ensure that the CEAG has a communication and engagement plan in place that includes

1. varied communication approaches to broaden reach and clearly communicate the CEAG's intentions and plans, including the corresponding benefits, to increase stakeholders' understanding and acceptance
2. adequate approaches to improve data users' awareness of and engagement with CEAG data, to further support CEAG tool usage and data interpretation, including working with users with less statistical capacity.

Management response

Management agrees with the recommendation.

1. The CEAG developed a communication and engagement plan for the 2021 cycle, which was approved by the CASC. Within budget constraints, the CEAG will work with partners in Census Communications to review and revamp this plan to create a new 2026 plan, in alignment with the AgZero initiative and its associated benefits. The new plan will explore varied communication approaches to broaden reach and outline the CEAG's benefits to increase stakeholders' and farmers' understanding and acceptance.
2. The 2021 CEAG developed a comprehensive dissemination plan with a vision to continue to meet user needs: more cross-cutting, horizontal stories and more outreach with Canadians via various media. The plan outlined key areas to enhance the CEAG data user experience: data products, data visualization products and mapping tools, analytical products, user guides, and webinars. Similarly, within the approved budget, a new enhanced 2026 dissemination plan will be created in collaboration with communication and dissemination service areas and be approved by the CASC.

A key focus of the plan will be to articulate approaches to improve data users' awareness of and engagement around CEAG data to further support CEAG tool usage and data interpretation, including working with users with less statistical capacity. These approaches could include giving targeted webinars, sending emails and using social media to announce releases, and consulting with users to better meet their needs.

Deliverables and timelines

A 2026 CEAG communications and engagement plan and a 2026 CEAG dissemination plan, developed in collaboration with relevant service areas and both approved by the CASC, will be delivered by September 2025 and December 2026, respectively.

Appendix A – Interview quantification scale

Interview responses are quantified and categorized in this report using the scale shown in the table below.

Appendix A – Examples of data needs on intersectionality

Term	Definition
One	One is used when one participant provided the answer.
Few	Few is used when 4% to 15% of participants responded with similar answers. The sentiment of the response was articulated by these participants but not by other participants.
Some	Some is used when 16% to 45% of participants responded with similar answers.
About half	About half is used when 46% to 55% of participants responded with similar answers.
Most or a majority	Most, or a majority, is used when 56% to 89% of participants responded with similar answers.
Almost all	Almost all is used when 90% to 99% of participants responded with similar answers.
All	All is used when 100% of participants responded with similar answers.

Statistics Canada has a mandate and legal obligation under the Statistics Act to conduct the Census of Agriculture (CEAG) every five years. The CEAG provides a comprehensive profile of the physical, economic, social and environmental aspects of Canada's agriculture industry. It is the only data source that consistently provides high-quality detailed statistical information on agriculture for small geographic areas and collects a wide range of data at the national, provincial and subprovincial levels.

The CEAG provides both a snapshot in time of the agriculture industry and changes over time that are essential to inform public and private decision making. Its expected outcomes are to provide high-quality data and information that is accessible, available and relevant to all data users^{Footnote 1}. In addition to achieving the CEAG outcomes, the 2021 CEAG was expected to

• decrease response burden on farm operators
• increase integration with Statistics Canada's centralized infrastructure
• increase harmonization between the methods, concepts and processes of the CEAG and other economic statistics programs
• publish more data without compromising the confidentiality of respondents
• align the business model with Statistics Canada's modernization objectives^{Footnote 2}.

The evaluation was conducted in accordance with the Treasury Board Policy on Results and Statistics Canada's Risk-Based Audit and Evaluation Plan (2023/2024 to 2027/2028). The objective of the evaluation is to provide credible and neutral information on the ongoing performance of the CEAG and to identify potential areas to consider for the CEAG's next cycle.

Key findings and recommendations

The CEAG program achieved its expected outcomes of providing high-quality data and information that is accessible, available and relevant to all data users with the timely delivery of the 2021 CEAG.

Compared with 2016, the 2021 CEAG both sustained and improved on various factors that contribute to producing high-quality data and to their relevance to users, their availability and their accessibility. The 2021 CEAG made progress on its objectives to improve operational efficiencies through integration with Statistics Canada infrastructure, increase harmonization with other economic statistics programs and reduce response burden. As with transformational changes, full potential benefits will be realized moving forward as the processes and activities become fully integrated and more familiar to staff.

In general, the main overarching planning processes for the 2021 CEAG were similar to the 2016 cycle, with the addition of several significant methodological and operational changes: the change in the farm definition, the migration to the Integrated Business Statistics Program, the continued development of the electronic questionnaire and adaptations made because of the pandemic. The risk management process contributed to achieving expected results, as well as mitigating the risks that are specific to the changes made in the 2021 cycle. The significant number of changes for the 2021 CEAG made planning more challenging because of some unpredictable implications from the changes, and staff turnover added additional planning constraints. These large-scale changes implemented for the 2021 CEAG have also fundamentally altered many processes, establishing a new baseline moving forward.

Overall, the 2021 CEAG communication strategy and activities were effective in maintaining, and in some cases improving, awareness of CEAG products. New and improved tools increased awareness and facilitated users' understanding of how to interpret CEAG data. Some areas noted for consideration for subsequent cycles of the CEAG are continuing to amplify the CEAG through various communication and engagement channels; empowering users to access, use and interpret the data; and providing clearer information and guidance on changes.

In light of these findings, the following recommendations are proposed:

Recommendation 1

The Assistant Chief Statistician (ACS), Economic Statistics (Field 5), should

1. ensure that the process in place to regularly review resources includes revisiting the balance between upcoming and planned deliverables (including those for communication and engagement) and the corresponding challenges versus available time and resources
2. in addition to the regular review process, given that the 2026 CEAG is not expected to include any new large-scale changes, verify that the planned levels and allocation of resources are correctly aligned with the new baseline resulting from the changes made for the 2021 CEAG; this will help ensure that future core activities are resourced appropriately.

Recommendation 2

The ACS, Economic Statistics (Field 5), should ensure that effective training, retention and succession plans are developed, reviewed regularly and aligned with future needs of the CEAG program.

Recommendation 3

The ACS, Economic Statistics (Field 5), should ensure that the CEAG has a communication and engagement plan in place that includes

1. varied communication approaches to broaden reach and clearly communicate the CEAG's intentions and plans, including the corresponding benefits, to increase stakeholders' understanding and acceptance
2. adequate approaches to improve data users' awareness of and engagement with CEAG data, to further support CEAG tool usage and data interpretation, including working with users with less statistical capacity.