The Consumer Price Index (CPI) measures price changes over a period of time for a "fixed basket"—or a fixed set—of goods and services consumed by Canadian households. Statistics Canada publishes CPI-related indexes on a monthly and annual basis. Other CPI data products, such as updates to the CPI reference manual, analytical and technical papers, and data visualizations, are disseminated at various frequencies throughout the year. The CPI is used by a number of public, private, and not-for-profit organizations, as well as by the media, researchers and individual Canadians, to monitor inflation, guide fiscal and monetary policies, index payments, and adjust contracts, among other things.

This evaluation was conducted by Statistics Canada in accordance with the Treasury Board's Policy on Results (2016) and Statistics Canada's Risk-Based Audit and Evaluation Plan (2019/2020 to 2023/2024). The primary objective of the evaluation was to provide a neutral, evidence-based analysis of the relevance of the CPI and its complementary indexes to municipal governments (MGs) and organizations supporting MGs; the effectiveness of communication strategies related to the CPI; and the extent to which the planning, design and implementation of innovation activities were effective and aligned with Statistics Canada's modernization initiative. It should be noted that the conducting phase of the evaluation, during which all data for the report were collected, was completed before the COVID-19 pandemic. As a result, the findings and recommendations do not consider the activities or decisions that took place after March 2020.

The evaluation found that Statistics Canada was the main source of CPI information for MGs and organizations supporting MGs. The CPI's related indexes, including the city indexes, were used at the municipal level for indexation, policy support, budgeting, reporting, and in some cases the creation of a municipal price index. There was limited use of the inter-city indexes by some users and interpretability was identified as an area for improvement—more details on concepts and the use of plain language would be beneficial. In terms of gaps, the evaluation found that more detailed geographic data and index categories were desired by interviewees. Program representatives indicated that they were aware of the gaps and that they continue to pursue alternative sources of reliable information to address them.

In terms of communication activities, the CPD is transitioning away from a dissemination culture focused primarily on traditional vehicles such as The Daily and the Integrated Metadatabase, towards a more strategic communication culture aimed at leveraging a variety of communication tools and channels – such as using social media to broaden its communications. Another example is the launch of the CPI portal in August 2019 which includes: the CPI Data Visualization Tool, recently released key indicators, a variety of featured products, direct access to CPI products, and information on new products and activities. To evaluate the effectiveness of new communication activities, the division will need to develop metrics and performance measures.

All projects and activities reviewed were aligned with the modernization initiative and expected results. They focussed on increasing the use of data from alternative sources, integrating data, and on building capacity. The CPD is sharing information internationally and across the agency, as well as leveraging best practices and lessons learned. Governance and risk management are in place to monitor projects and the implementation of an agile approach (e.g., "network of teams") was effective, however, governance and risk management practices for activities could be improved. Finally, performance measurement could be strengthened.

The evaluation proposed three recommendations:

• Recommendation 1:
  Given the CPD's desire to become more strategic with its communications activities, the division engage corporate communications to improve its knowledge of the new communications tools available. Furthermore, the division should update its communication planning process to include the use of measurement tools (in addition to web metrics) to gather information on the effectiveness of activities. This information will help improve activities such as the communication of program changes to users.
• Recommendation 2:
  The CPD strengthen its performance measurement framework process for its DPMF projects to ensure that project outcomes are clearly articulated and that indicators are balanced, comprehensive and aligned with the identified outcomes and expected benefits.
• Recommendation 3:
  The CPD include all activities that have a significant impact on the DPMF projects, such as the high-capacity processing infrastructure activities, within the governance and oversight structures of the DPMF projects. The adoption of this more holistic approach will improve overall coordination and risk management.

Linkage of the Longitudinal Immigration Database (IMDB) to the Labour Force Survey (LFS) (001-2020)

Linkage of Statistics Canada's Research Data Centre Longitudinal Worker File (LWF) with the Employment Insurance EI Status Vector File (EISV) (002-2020)

Temporary Foreign Worker and employment/earnings dynamics (003-2020)

Dynamic Retirement Income Modelling Linkage Environment (004-2020)

Linkage of Business Accelerators and Incubators and Peer to Peer lending databases to the Linkable File Environment (LFE) (005-2020)

Innovation and Growth in the Canadian Life Sciences Industry (007-2020)

Linkage of the Trade Show stream program data to the Linkable File Environment (008-2020)

Linkage of graduate student administrative data (including funding information from the Natural Sciences and Engineering Research Council, the Social Sciences and Humanities Research Council, and the Canadian Institutes of Health Research; collectively known as 'Tri-Agency') to the T1 Family File (009-2020)

Gross Flows of Workers Into and Out of Industries in New Brunswick, 2000 to 2017. (010-2020)

Linkage of the T3010 and T5013 tax forms to the Linkable File Environment (011-2020)

Canadian Perspectives Survey Series (CPSS) (012-2020)

Estimating agricultural employees using administrative data (013-2020)

Linkage of the Canada Emergency Business Account borrower data file to the Linkable File Environment (014-2020)

Linkage of the Canadian Survey on Business Conditions (CSBC) to the Linkable File Environment ( LFE) (015-2020)

Provisional causes of death by subpopulations: Focus on COVID-19 (016-2020)

Creation of the National Accounts Longitudinal Microdata File (NALMF) (017-2020)

The Imperial Oil Limited Worker Health Study: cancer and mortality follow-up study (Cohort V) (018-2020)

Official language minorities businesses in Canada – definitions and implications (019-2020)

Linkage of the Labour Force Survey (LFS) to census and immigration data (020-2020)

The entrepreneurial activities of lottery winners in Canada (021-2020)

Linkage of the Office of the Superintendent of Bankruptcy's Consumer Proposal data to Statistics Canada's Canadian Employer-Employee Dynamics Database (022-2020)

Experimental estimation of month-by-month income from the Labour Force Survey (023-2020)

Short and long term income of cancer survivors in Canada – Population-based analysis using linked data (024-2020)

Corporate Concentration, Investment, and Labour Demand (025-2020)

Long-Term Mortality (1950–2018) and Incidence (1969–2017): Follow-up of the Canadian Fluoroscopy Cohort Study (CFCS). (026-2020)

Ontario Opioid Data Collection and Community Response Project: Linking Simcoe-Muskoka Opioids data with Census, income, health, tax, immigration and criminal courts data (027-2020)

Linkage of the 2017 Canadian Survey on Disability (CSD) and Tax data (028-2020)

Linkage of the National Research Council Industrial Research Assistance Program (NRC-IRAP) data file to the Linkable File Environment (LFE) – (029-2020)

Elections Canada Data Quality Confirmation Study of the National Register of Electors (030-2020)

Linkage of the Canada Emergency Wage Subsidy (CEWS) claims microdata files to the Linkable File Environment (031-2020)

Evaluating the Causal Impact of the Employment Equity Act on the Firm-Level Representation of Women (032-2020)

Veterans Survivors Quantitative Study Data integration for estimating the population of spouses and survivors who married or entered into a common-law relationship with a Veteran on or after the Veteran's 60th birthday (034-2020)

Longitudinal and Intersectional Measures Drug Overdose Analytical Data Project (035-2020)

2006 Aboriginal Peoples Survey (APS) and 2016 Census Data Integration (036-2020)

Microdata Linkage for the Canadian Fishing Fleet Cost and Earnings Study – Phase 2 (037-2020).

British Columbia COVID-19 testing by subpopulations (038-2020)

Linkage of Employment Insurance Statistics - Monthly to the Record of Employment (039-2020)

Using administrative data to assist in processing data from non-respondent households to the Census of Population, at the imputation stage, once field collection is completed. (040-2020)

Postsecondary and Workplace Skills (PAWS) Project (041-2020)

Evaluation Report

December 2020

• The report in short
• What is covered
• What we learned
  • User experience
  • Design, delivery and implementation
• How to improve the program
• Management response and action plan
• Appendix A – Evaluation types
• Appendix B – Virtual Data Lab business case
• Appendix C – Business outcome indicators and performance measurement method
• Appendix D – Statistics Canada's modernization initiative
• Appendix E – Contingency plans

The report in short

The Data Access Division (DAD) is responsible for providing microdata access to researchers outside Statistics Canada and for maintaining the repository of all deemed employees of the agency. This includes providing support, expertise and standards of data provision to subject-matter areas within Statistics Canada, and ensuring that the confidentiality of information is protected in all microdata access agreements.

Building on the foundation of the research data centres (RDCs) and the Canadian Centre for Data Development and Economic Research (CDER), Statistics Canada created the Virtual Data Lab (VDL) pilot that—in its end state—will provide remote access to detailed anonymized social and business microdata for research and analysis through a secure cloud-based interface. The VDL aims to increase the number of researchers that use Canadian data, increase the number of data holdings that users can access, modernize existing IT infrastructure, develop shared-risk partnerships and move existing operations to a cloud-based architecture.

The evaluation was conducted by Statistics Canada's Evaluation Division in accordance with the Treasury Board's Policy on Results (2016) and Statistics Canada's Risk-Based Audit and Evaluation Plan (2019–20 to 2023–24). The objective of the evaluation was to provide a neutral, evidence-based assessment of program alignment with identified user needs. The evaluation reviewed whether existing VDL assessments conducted within the agency were adequately comprehensive in their coverage, and whether findings and recommendations from these assessments were integrated into the design, delivery, strategic planning and implementation of the VDL project. In addition, the evaluation assessed the extent to which performance measurement and risk assessment frameworks have been implemented.

The evaluation methodology consisted of a document review, key informant interviews with Statistics Canada professionals working in the DAD and Digital Solutions Field, and additional lines of inquiry, where applicable.

It should be noted that the conducting phase of the evaluation, during which all data for the report were collected, was completed before the COVID-19 pandemic. As a result, the findings and recommendations do not consider the activities or decisions that took place after March 2020.

Key findings and recommendations

Recommendation 1

The Assistant Chief Statistician, Strategic Engagement (Field 4), should ensure that given the level of complexity and dependence on other parts of the agency, contingencies and clearer timeframes be articulated for activities.

Recommendation 2

The Assistant Chief Statistician, Strategic Engagement (Field 4), should ensure that governance mechanisms are in place that effectively manage horizontal activities in a holistic manner including the clear establishment and understanding of roles and responsibilities.

What is covered

The evaluation was conducted in accordance with Treasury Board's Policy on Results(2016) and Statistics Canada's Integrated Risk-based Audit and Evaluation Plan (2019–20 to 2023–24). In support of decision-making, accountability and improvement, the main objective of the evaluation was to provide a neutral, evidence-based assessment of the Virtual Data Lab (VDL) pilot. Because of the status of the project at the time of the evaluation, a formative approach was employed.^{Footnote 1}

Data Access Division

The Data Access Division (DAD) is responsible for governing access to confidential microdata to researchers—and deemed employees—outside and within Statistics Canada. This includes providing support, expertise and standards of data provision to subject-matter areas within the agency and ensuring that the confidentiality of the information is protected in all microdata access agreements. The DAD delivers microdata access through various programs based on the type of data requested, user-specific needs, the purpose of accessing the dataset and the sensitivity level of the requested data file. The provision of microdata access is then administered through a continuum in accordance with organizational security requirements and the sensitivity of the data files.

As part of Statistics Canada's ongoing efforts to support its modernization agenda (Appendix D), it developed the Confidentiality Classification Tool (CCT)—a corporate tool for classifying confidential data. The CCT is a short questionnaire that generates a standard confidentiality classification along a continuum of risk (a score from 0 to 9). CCT scores are used to determine the nature and degree of confidentiality protections Statistics Canada employs in its information holdings. In particular, this tool is used to determine access conditions for trusted external data users, such as VDL partner organizations and researchers, through an access management framework adapted from the Five Safes framework developed by the United Kingdom's Office for National Statistics (this is a holistic approach to data protection that goes beyond information technology [IT] and physical site protection).

Virtual Data Lab pilot project

Building on the foundation of the research data centres (RDC) and the Canadian Centre for Data Development and Economic Research (CDER), Statistics Canada created the VDL pilot that—in its end state—will provide remote access to detailed, anonymized social and business microdata for research and analysis through a secure cloud-based interface. The VDL business case outlined the following key outcomes (Appendix B):

• Enhance UX when accessing anonymized microdata.
• Leverage new technology, methods and data.
• Apply risk management practices (to optimize data accessibility with protective measures).
• Increase collaboration and partnerships between data users and providers.
• Build analytical and research capacity.
• Support stronger evidence-based decision-making.

The first prototype of the VDL is the virtual Federal Research Data Centre, which is located at the Canada Mortgage and Housing Corporation (CMHC) headquarters in Ottawa. With new partnerships being developed, this shared-risk model will be expanded to additional partner agencies and their users. The CMHC facility—where deemed employees are provided secure access to anonymized data—is being used to test UX, access and security protocol, starting with a designated certified room similar to an RDC and moving into regulated access for approved CMHC employees to use anonymized research files in separate authorized workspaces. The VDL pilot will evolve gradually toward remote cloud-based access once Statistics Canada's cloud infrastructure becomes available.

The anonymized data used by CMHC researchers are housed on secure Statistics Canada servers and authorized researchers are required to use Statistics Canada secure encrypted devices to connect to these central servers through Virtual Desktop Infrastructure. Less sensitive (i.e., with a lower CCT score) information may be accessed in authorized workspaces, while more sensitive anonymized data (CCT 8) are accessible only within the designated certified room.

The VDL project will also be expanding to include the Bank of Canada (BoC) and the Health Portfolio (Opioid Pathfinder Project), with additional partnerships planned.

Description of Figure 2 - Location of access and file sensitivity for deemed employees

The figure depicts the types of locations of access mapped against the accreditation level and file sensitivity level (Confidentiality Classification Tool [CCT] score).

In the figure, the file sensitivity (CCT score) ranges from 1 to 9 with 1 being the lowest and 9 the highest.

The accreditation level ranges from level 0 to level 4 with level 0 being the lowest.

Location access includes open office (cubicle environment), closed office (personal office or conference room), physical enclave (designated certified room including virtual federal research data centre, research data centre, and federal research data centre).

In terms of location access:

• A accreditation level of 0 provides no access regardless the file sensitivity score
• For file sensitivity scores of 9, there is no access outside of Statistics Canada headquarters regardless accreditation level
• Level 1 accreditation means access via physical enclave for file sensitivity scores of 1 to 8
• Level 2 accreditation means access via open office for file sensitivity scores 4 and below, closed office for file sensitivity scores 5 to 7, and physical enclave for a file sensitivity score of 8
• Level 3 accreditation means access via open office for file sensitivity scores 5 and below, closed office for file sensitivity scores 6 and 7, and physical enclave for a file sensitivity score of 8
• Level 4 accreditation means access via open office (experimental research) for file sensitivity scores of 5 and below, closed office for file sensitivity scores 6 and 7, and physical enclave for a file sensitivity score of 8

The highest CCT score of the datasets associated with a project dictates the mode and location of access.

The length and complexity of the process to apply for access to Statistics Canada microdata files, as well as the limitations on use and outputs, are dependent on the sensitivity level (CCT score) of the data file to which access has been requested (and granted) and—therefore—vary by access program. The RDC, CDER and VDL programs provide access to more sensitive data files, so application, access and use protocols are similar across these programs.

Progress toward the implementation of a cloud-based platform for VDL users is contingent on progress made by other Statistics Canada areas, namely the Data Analytics as a Service (DAaaS) and IT teams, as well as partner agencies and cloud services. With relatively short timelines, triangulated efforts made by all teams involved must continue to properly test the product and UX, minimize risks, and work toward key project management timelines, deliverables and thresholds.

In summary, the VDL is leveraging new and innovative tools, as well as testing new delivery platforms and formats. The project is based on an incremental approach to project development and leveraging lessons learned to adapt and improve quickly.

Evaluation

The scope of the evaluation was established based on a document review and meetings with DAD experts. The following areas were identified for review:

Guided by a formative evaluation approach, two main collection methods were used:

• Internal stakeholder interviews
  Semi-structured interviews with the DAD and Digital Solutions
• Document review
  Review of internal agency documents

Four main limitations were identified, and mitigation strategies were employed:

The conducting phase of the evaluation, during which all data for the report were collected, was completed before the COVID-19 pandemic. As a result, the findings and recommendations do not consider the activities or decisions that took place after March 2020.

What we learned

User experience

The assessments are comprehensive in their coverage of existing users and in identifying user needs and feedback trends that can be used to inform strategic planning, particularly among federal government and academic users.

The large majority of existing user feedback was sourced from federal government researchers. Additional insights were provided by RDC employees who were able to confirm that the trends identified in existing assessments align with those of RDC user feedback (mainly academic users and government employees). Although these assessments are representative of current microdata users, future user needs assessments should be expanded to cover a broader range of users and identify project opportunities and prospective partners.

For improved understanding of VDL user requirements, the team will need to extend its coverage of user consultations to provincial and territorial governments to assess the unique needs of these key potential user groups, in adherence with the intended business outcome of establishing new and increased data partnerships with other government departments. In the longer term, consulting private industry and civil society organizations would provide additional insights on the user needs of these groups, in alignment with the long-run business outcome of increasing the number of researchers that use Canadian data and microdata files.

Collectively, existing assessments are comprehensive in their coverage of all project dimensions and aligned with the desired outcomes set out in project planning and framework documents.

A review of the existing assessments found that the user process was covered effectively. Information was gathered on all seven steps, beginning with the proposal and ending with the vetting.

The evaluation team identified areas of the microdata access process in which trends in user feedback were consistent, including increasing data discoverability, simplifying the proposal process, implementing modern and user-centric tools and support mechanisms, allowing collaboration between researchers within and across research projects, and improving the vetting process. These key areas were examined further in interviews with experts from the VDL team and the DAD.

In terms of alignment with the VDL desired outcomes, the evaluation found that the assessments covered four key outcomes sufficiently:

• Enhance UX when accessing anonymized microdata.
• Leverage new technology, methods and data.
• Apply risk management practices (to optimize data accessibility with protective measures).
• Increase collaboration and partnerships between data users and providers.

Design, delivery and implementation

The evaluation considered whether findings of all assessments were leveraged to inform decision-making, project management and strategic planning, including the degree to which they were integrated into the development of tools and processes for the VDL. Assessment findings and recommendations have been integrated into VDL plans to a significant degree.

Assessments of UX, privacy and security have been leveraged to inform strategies and planning. However, timelines for delivery were not always clear.

The evaluation found that the VDL team—and the DAD more generally—has made a significant effort to consult users and test products prior to implementation. Assessments of user needs and UX research were leveraged to develop business requirements for the new MAP and Client Relationship Management System (CRMS). During the planning stages, the VDL team also performed an international scan to identify best practices that could be leveraged for the project, its services and framework.

Based on the UX assessments and discussions with RDC experts, the evaluation team was able to identify recurring trends in user feedback across the microdata access process. These have been further articulated below, including planned and proposed solutions.

Overall, the evaluation found that the VDL team worked collaboratively with the DAaaS team to identify trends in UX and possible solutions (HMW statements). Although these statements were beneficial in presenting possible solutions, they were not included in project planning, nor were timelines established for their integration. An additional area for improvement in UX could come from formalized lessons learned over the first few months of the project rollout, especially as new partners are onboarded.

Lastly, because of the complex integrated nature of the project, timelines are heavily reliant on other areas of the agency. Some activities related to the MAP and cloud services are scheduled to take place in the near term, while others have been pushed to future iterations of the project. The timing of those activities and items being pushed was generally unclear.

The DAD performed an international scan of NSOs with similar capabilities and programs to determine best practices and assess similar program offerings worldwide.

The approach taken by the VDL team was developed by completing a point-in-time international scan of similar offerings among other NSOs. The use of the Five Safes framework came from this scan. Best practices of the NSOs in Australia, Denmark, Finland, France, Germany, Sweden and the United Kingdom were assessed when developing the possible format of the VDL, and its services align with international best practices from this point in time.

It would be beneficial for the VDL team to establish an international network of microdata access experts to share best practices proactively to replace point-in-time international scans that may miss key elements. In this way, the VDL team would remain informed of new and innovative approaches to microdata access that could be used or adopted.

Performance indicators have been articulated and work to improve them is ongoing.

Members of the VDL team noted that, while the performance measurement framework and indicators are outlined in the business case and project plan, they should be viewed as preliminary (Appendix C). At the time of the evaluation, the DAD was set to begin working with internal performance measurement experts to develop divisional performance indicators. It was unclear whether this would include additional KPIs for the VDL project. The VDL team could work proactively with other DAD project teams to identify KPIs that balance outcomes at the divisional level while also monitoring the progress of the VDL project. In terms of monitoring, there was little indication that regular monitoring of progress against the indicators in the project plan was taking place.

A risk management framework exists and active risk management is taking place. Contingency planning and a clarification of responsibilities would be useful.

A comprehensive review of the business process and risk framework was completed in 2019 and documentation and regular reporting on risks take place through the Departmental Project Management Framework. Members of the VDL team noted that they will take an iterative approach to amending risks and proposed mitigations or response strategies as changes arise within the project and as they learn from partners and users, although no formal documentation was provided.

Risk management includes effective contingency planning as part of mitigation development. The evaluation found that the articulation of contingency plans could be improved. Some examples where such plans would be beneficial include the proposal and vetting processes (Appendix E). For example, changes to the proposal process are highly dependent on both a new CRMS and MAP. However, no contingencies were found in the case of either of them being delayed or not meeting requirements. For the vetting process, a standard operating procedure could be established in the short term to improve consistency and clarity.

Lastly, the evaluation found that a clarification of responsibilities would be useful, which is particularly important given the complexity of the project and the strong dependency on partners. In particular, it was noted that, while there was a difference between what was considered a VDL project responsibility and what was considered a DAD responsibility, it was unclear how and when things were intended to be shared, who was ultimately making decisions, and how the work would be distributed.

Performance measurement and risk management

• The Data Access Division (DAD) and the virtual data lab (VDL) team will work with internal performance measurement experts to develop and implement a logic model.
• As part of existing assessments, a comprehensive review of security procedures and processes of the virtual Federal Research Data Centre (vFRDC) was completed.
• Service-centric metrics and reporting were identified as a mandatory and high-priority business requirement for a new Client Relationship Management System (CRMS).
• The DAD will take a responsive approach to amending policies and directives as needs arise. This includes amending the risk management framework to reflect risks from the transition to remote access.

How to improve the program

Management response and action plan

Recommendation 1

The Assistant Chief Statistician, Strategic Engagement (Field 4), should ensure that given the level of complexity and dependence on other parts of the agency, contingencies and clearer timeframes be articulated for activities.

Management response

Management agrees with the recommendation.

The VDL project will maintain evergreen plans, schedules and contingencies. An enhanced tracking process will be implemented to ensure plans, schedules, risks and contingencies for VDL project activities and deliverables are complete, continually updated, and regularly reviewed with service providers. These will be reported on using Departmental Project Management Framework tools such as the monthly Dashboard, and monthly Interdependency Report to ensure clear timeframes are articulated across the agency.

In collaboration with service partners, the VDL project will continue to report to committees (such as the Strategic Management Committee [SMC]) in order that interdependencies, risks, contingencies and issues be shared and addressed with other parts of the agency.

Deliverables and timelines

The Assistant Chief Statistician, Strategic Engagement (Field 4), will ensure the delivery of:

• An enhanced tracking process for plans, schedules, risks and contingencies. (July 2021 – Director General [DG] Data Access and Dissemination Branch)

Recommendation 2

The Assistant Chief Statistician, Strategic Engagement (Field 4), should ensure that governance mechanisms are in place that effectively manage horizontal activities in a holistic manner including the clear establishment and understanding of roles and responsibilities.

Management response

Management agrees with the recommendation.

The existing governance structure and processes for the VDL for the regular reporting of progress, planning of upcoming activities, reviewing of roles and responsibilities, and evaluating of risks will be formalized to ensure they are complete and conducted in a holistic horizontal manner; the updated structure will be presented to a senior level committee such as SMC. The VDL project will regularly report progress through governance and oversight mechanisms to ensure issues are addressed consistently, effectively, and holistically.

The VDL project will continue to participate in a number of multi-divisional working groups and committees where horizontal activities can be discussed. In collaboration with service partners, a complete review of roles and responsibilities for all remaining key activities will be undertaken. Findings will be reported to senior management committees such as the SMC, where changes can be approved. The effectiveness of the redefined roles and responsibilities will be evaluated through the pilots and Service Level Agreements based on the results will be prepared before the launch of the VDL in production.

Deliverables and timelines

The Assistant Chief Statistician, Strategic Engagement (Field 4), will ensure the delivery of:

1. A formalized governance structure and processes for the VDL process. (June 2021 - DG Data Access and Dissemination Branch)
2. A completed review of roles and responsibilities for the VDL project. (July 2021 - DG Data Access and Dissemination Branch)
3. Completed Service Level Agreements. (October 2021 - DG Data Access and Dissemination Branch)

Appendix A

Appendix B – Virtual Data Lab business case

The historical background for this project

In 2017, Statistics Canada developed a modernization vision, which included an expansion of access to social and business microdata and administrative data. The approved direction is to develop new ways of accessing microdata (e.g., through virtual access), modernize existing tools that support access to microdata and make more types of data available to the user.

In modernizing our access to microdata, this project plans to

• enhance UX when accessing anonymized microdata
• leverage new technology, methods and data
• apply risk management practices (to optimize data accessibility with protective measures)
• increase collaboration and partnerships between data users and providers
• build analytical and research capacity
• support stronger evidence-based decision-making.

The overall business context into which the project fits

Research communities have expressed the need to better access Statistics Canada data holdings. As we improve data access, we are

• enhancing users' experience when working with anonymized microdata
• harnessing new technologies
• changing the culture in terms of our risk measures, reducing barriers and reconsidering how we can share risk with our partners
• increasing our collaborative efforts and increasing our analytical and research capacities
• providing more types of data and more integrated datasets to researchers and policy makers, and allowing for stronger evidence-based decision making and policy making.

In increasing data access, our objectives are to

• facilitate its use for leading-edge research and analysis across the country
• drive innovation and inclusion
• support evidence-based decision-making and policy-making
• better enable the mobilization of data across the federal government using Statistics Canada's expertise and capacity.

The drivers that triggered the change

Because of advances in technology, the current access programs no longer meet the needs of researchers. Existing tools are becoming outdated and need to be modernized. Our metadata systems also need to be strengthened and modernized.

Appendix C – Business outcome indicators and performance measurement method

Appendix D – Statistics Canada's modernization initiative

The Vision: A Data-driven Society and Economy

Modernizing Statistics Canada's workplace culture and its approach to collecting and producing statistics will result in greater and faster access to needed statistical products for Canadians. Specifically, the initiative and its projects will:

• Ensure more timely and responsive statistics – Ensuring Canadians have the data they need when they need it!
• Provide leadership in stewardship of the Government of Canada's data asset: Improve and increase alignment and collaboration with counterparts at all levels of government as well as private sector and regulatory bodies to create a whole of government, integrated approach to collection, sharing, analysis and use of data
• Raise the awareness of Statistics Canada's data and provide seamless access
• Develop and release more granular statistics to ensure Canadians have the detailed information they need to make the best possible decisions.

The Pillars

User-Centric Delivery Service:

• Users have the information/data they need, when they need it, in the way they want to access it, with the tools and knowledge to make full use of it.
• User-centric focus is embedded in Statistics Canada's culture.

Leading-edge Methods and Data Integration:

• Access to new or untapped data modify the role of surveys.
• Greater reliance on modelling and integration capacity through R&D environment.

Statistical Capacity Building and Leadership:

• Whole of government, integrated approach to collection, sharing, analysis and use of data.
• Statistics Canada is the leader identifying, building and fostering savvy information and critical analysis skills beyond our own perimeters.

Sharing and Collaboration:

• Program and services are delivered taking a coordinated approach with partners and stakeholders.
• Partnerships allow for open sharing of data, expertise and best practices.
• Barriers to accessing data are removed.

Modern Workforce and Flexible Workplace:

• Organization is agile, flexible and responsive to client needs.
• Have the talent and environment required to fulfill our current business needs and be open and nimble to continue to position ourselves for the future.

Expected Outcome

Modern and Flexible Operations: Reduced costs to industry, streamlined internal processes and improved efficiency/support of existing and new activities.

Appendix E – Contingency plans

• Proposal process: Data Analytics as a Service's "how might we" statements (developed in collaboration with the Virtual Data Lab [VDL] team) and research data centre (RDC) experts noted this as a challenge that should be overcome for VDL users. This sentiment was echoed by the business outcome indicators of the VDL project plan, according to which the project should facilitate faster and timelier onboarding of researchers to the VDL (a goal of one week of onboarding within the first six months of the VDL).
  • Although the proposal process itself was identified as a challenge, the discoverability of data and variables included in the files for which researchers are seeking access was also noted as an obstacle to developing a proposal. Although this challenge has been partly resolved through collaboration with subject-matter areas (including variables in the Integrated Metadatabase), contingency plans for providing this service were not identified. Data discoverability improvements are contingent on the acquisition of a new Client Relationship Management System that includes these capabilities, for which there was no clear timeline. No secondary strategy was identified.
  • Many administrative challenges related to the proposal process are intended to be overcome through the implementation of the Microdata Access Portal. However, no contingency plan was identified in the event of delivery being delayed or specifications being changed.
  • The project plan outlines an intended onboarding process of one week per user, although monitoring of progress toward this and other business indicators was not identified. It would be beneficial to begin monitoring indicators laid out in the project plan as a means to reprioritize and provide clarity on project timelines and intended outcomes.
• Vetting process: The VDL business case suggested that the VDL will "grant researchers and administrative personnel access to a variety of modern tools such as analytical software, visualization and client relations management tools that will have the appropriate monitoring controls and vetting procedures built-in." Much like the proposal process, users expressed challenges with the existing vetting process. This feedback was consistent across the Canadian Research Data Centres Network, but was also included in initial feedback received from Canada Mortgage and Housing Corporation researchers on the VDL pilot.
  • Addressing the current vetting process was not identified as a short-term priority. It was noted that the process could be enhanced by implementing automated or built-in vetting tools (e.g., the Output Checker Workflow Application, which was being analyzed at the time of the evaluation). In the shorter term, the VDL team could work with other areas of the Data Access Division to implement a standard operating procedure to ensure that—at a bare minimum—the consistency and clarity of vetting training would be improved.

Form 91Q

Confidential when completed.

This information is collected under the authority of the Statistics Act. R.S.C., 1985, c. S-19

Control use

SSID

• Prov.
• CD No.
• CU No.
• VR Line No.

Contact person

Office Use Only

Result of interview

1. Completed questionnaire
2. Incomplete questionnaire

Section I — Address or Exact Location of This Dwelling

Transcribe from the Assignment List (Form 1B)

• Street and No. or lot and concession
• Apt. No.
• City, town, village
• Province/territory
• Postal Code
• AD

Section II — Verification of Dwelling

Interviewer check item:
1. Is there a dwelling (a set of living quarters with a private entrance) at the address listed above?

1. Yes; Continue with Question 2
2. No; What is located at this address?
   3. Business or professional office of some sort (e.g., dentist office, gas station); Continue with Question 2
   4. Dwelling under construction; Continue with Question 2
   5. Dwelling demolished; Continue with Question 2
   6. Empty lot; Continue with Question 2
   7. Could not locate address; End interview
   8. Apartment no longer used as a separate dwelling; Continue with Question 2
   9. Other – Specify; End interview

Read script: refer to Form 91R
2. On Census Day, Tuesday May 11, was there a single set of living quarters at this address, or was there more than one?

1. None; End interview
2. One; Continue with Question 3
3. More than one; Did each have a private entrance?
   3. Yes; Go to Question 3 and complete a separate questionnaire for each dwelling
   4. No; Go to Question 3

Interviewer check item:
3. Identify person contacted.

• Family name:
• Given name and initial(s)

1. Occupant
2. Neighbour
3. Superintendent or building manager
4. Other; Specify

Section III — Dwelling Occupancy Status on May 11, 2021

4. Was someone living in the dwelling on Census Day?

1. Yes; Continue with Question 5
2. No; Go to Question 6
3. Don't know; End interview and find another contact

• If the dwelling is now occupied but the occupancy on May 11, 2021 is unknown, check "Don't know".
• Only check "Yes" or "No" based on the occupancy on Census Day, Tuesday, May 11, 2021.

5. On Census Day, were they living in the dwelling on a temporary or occasional basis, or was it their usual home?
A temporary or occasional basis would include such things as staying at a summer home or a second home.

1. Temporary – Specify; Go to Question 15
2. Usual Home; Go to Question 17
3. Don't know; Go to Question 17

6. Is the dwelling generally occupied on a temporary or occasional basis, or is it someone's usual home?
A temporary or occasional basis would include such things as a summer home or a second home.

1. Temporary – Specify; Go to Question 15
2. Usual Home; Continue with Question 7
3. Don't know; Continue with Question 7

7. Were the usual residents temporarily away, or staying outside of Canada on Census Day, Tuesday, May 11?
Temporarily away includes being away on business, at a summer home, on vacation, or at school.

1. Yes – Specify; Go to Question 17
2. No; Continue with Question 8
3. Don't know; Continue with Question 8

Section IV— Interview A — Dwelling Unoccupied on May 11, 2021

8. Was anyone living in the dwelling at any time between May 1st and Census Day, Tuesday, May 11?

1. Yes; Continue with Question 9
2. No; Go to Question 11
3. Don't know; Go to Question 11

9. Were they living in this dwelling on a temporary or occasional basis, or was it their usual home?
A temporary or occasional basis would include such things as staying at a summer home or a second home.

1. Temporary; Go to Question 15
2. Usual home; Continue with Question 10
3. Don't know; Go to Question 11

10. Could you tell me when these former occupants moved out of the dwelling?

1. On or before May 10, 2021; Continue with Question 11
2. On or after May 11, 2021; Return to Question 4, and obtain information about the dwelling for Census Day. Use a new questionnaire if necessary
3. Don't know; Continue with Question 11

Interviewer check item:
11. Is the person being interviewed an occupant of the dwelling listed in SECTION I?

1. Yes; Continue with Question 12
2. No; Go to Question 13

12. On what date did your household move into this dwelling?

1. On or before May 11, 2021; Return to Question 4, and obtain information about the dwelling for Census Day. Use a new questionnaire if necessary.
2. On or after May 12, 2021; Go to Question 15

13. Is someone currently living in the dwelling?

1. Yes; Continue with Question 14
2. No; Go to Question 15
3. Don't know; Go to Question 15

14. On what date did the current occupant(s) move into the dwelling?

1. On or before May 11, 2021; Return to Question 4, and obtain information about the dwelling for Census Day. Use a new questionnaire if necessary.
2. On or after May 12, 2021; Continue with Question 15
3. Don't know; Continue with Question 15

15. Was this dwelling suitable for year-round occupancy on Census Day, Tuesday, May 11?
That is, did it have a source of heat or power, and provide complete shelter from the elements?

1. Yes; Continue with Question 16
2. No; Continue with Question 16
3. Don't know; Continue with Question 16

16. Was this dwelling under construction or major renovation on Census Day, Tuesday, May 11?

1. Yes; End interview and complete Question 24 and Question 25
2. No; End interview and complete Question 24 and Question 25
3. Don't know; End interview and complete Question 24 and Question 25

Section V — Interview B — Dwelling Occupied on May 11, 2021

17. How many persons were living in the dwelling on Census Day, Tuesday, May 11?

Include:

• All persons who had their main residence at this address on May 11, 2021, including newborn babies, room-mates and person who were temporarily away,
• Canadian citizens, landed immigrants (permanent residents), persons asking for refugee status (refugee claimants), persons from another country with a work or study permit and family members living here with them,
• Persons staying at this address temporarily on May 11, 2021 who have no main residence elsewhere.

Exclude:

• Visitors who had their main residence elsewhere in Canada,
• Government representatives of another country or members of the Armed Forces of another country and their families,
• Residents of another country visiting Canada, for example, on a business trip or on vacation.

1. number of persons; Continue with Question 18
   If "00" persons (meaning ALL persons living in this dwelling are in the 'Exclude' group) End interview and complete Question 24 and Question 25
2. Don't know; Continue with Question 18

18. When did these people move into this dwelling?

1. On or before May 11, 2021; Go to Question 21
2. On or after May 12, 2021; Continue with Question 19
3. Don't know; Go to Question 23

19. Did anyone live in the dwelling prior to these people?

1. Yes; Continue with Question 20
2. No; Return to Question 4, and obtain information about the dwelling for Census Day. Use a new questionnaire if necessary.
3. Don't know; Go to Question 23

20. When did these former occupants move out of this dwelling?

1. On or before May 10, 2021; Return to Question 4, and obtain information about the dwelling for Census Day. Use a new questionnaire if necessary
2. On or after May 11, 2021; Return to Question 17 and obtain information for the May 11 occupants
3. Don't know; Go to Question 23

21. Do these people still live in the dwelling?

1. Yes; Go to Question 23
2. No; Continue with Question 22
3. Don't know; Go to Question 23

22. When did these people move out of the dwelling?

1. On or before May 10, 2021; Return to Question 4, and obtain information about the dwelling for Census Day. Use a new questionnaire if necessary.
2. On or after May 11, 2021; Go to Question 23
3. Don't know; Go to Question 23

23. What is the sex and age of each person usually living in the dwelling on Census Day, May 11?

Interviewer instructions:
Refer to Question 17 to obtain the total number of persons.
If Question 17 has no response or the number of persons is "00" or more than "06", End interview and complete Question 24 and Question 25

1. Number of persons

Obtain the sex and age for each person.
If the age of a person is unknown, an approximate age is acceptable.

Section VI — Classification of Dwelling

Interviewer:
To be completed by interviewer upon completion of interview.

24. What is the "dwelling type" of the dwelling listed in Section I? Mark one circle only.
For a list of dwelling types and their definitions refer to page 6.

1. Single-detached house
2. Semi-detached house
3. Row house
4. Apartment or flat in a duplex
5. Apartment in a building that has five or more storeys
6. Apartment in a building that has fewer than five storeys
7. Other single-attached house
8. Mobile home
9. Other movable dwelling

25. Is the dwelling listed in Section l suitable for year round occupancy?
That is, does it have a source of heat or power, and provide complete shelter from the elements?

1. Yes
2. No
3. Don't know

Section VII — Situations in the Field

Interviewer check item:
Check all that apply. Explain situations in Section VIII - Comments.

1. "No dwelling exists" or "Could not locate address" - explain the situation
2. More than one dwelling at the same address – explain the situation & write down the exact number of dwellings at this address
3. Two addresses describe the same dwelling (i.e. AD=01) – write down the SSID for each dwelling
4. Only one of the two addresses associated with the dwelling is listed on the Assignment List, Form 1B, you will conduct one interview for the entire dwelling (i.e. AD=02) – write the other address associated with the dwelling
5. Dwelling is a business or collective dwelling with a private dwelling at the address – explain the situation
6. Refusal by the occupant
7. Refusal by NON-occupant
8. Received a completed Form 2A during DCS
9. Other – explain the situation

Section VIII — Comments

(Space for comments)

Private dwelling type codes - Definitions

1. Single-detached house

A single dwelling not attached to any other dwelling or structure (except its own garage or shed).

A single-detached house has open space on all sides, and has no dwellings either above it or below it.

A mobile home fixed permanently to a foundation should be coded as a single-detached house. (See Code 8.)

2. Semi-detached house

One of two dwellings attached side by side (or back to back) to each other, but not attached to any other dwelling or structure (except its own garage or shed). A semi-detached house has no dwellings either above it or below it and the two units, together, have open space on all sides.

3. Row house

One of three or more dwellings joined side by side (or occasionally side to back), such as a townhouse or garden home, but not having any other dwellings either above it or below. If townhouses are attached to high-rise buildings, assign Code 3 to each townhouse.

4. Apartment or flat in a duplex

One of two dwellings, located one above the other. If duplexes are attached to triplexes or other duplexes or to other non-residential structures (e.g., a store), assign Code 4 to each apartment or flat in the duplexes.

5. Apartment in a building that has five or more storeys

A dwelling unit in a high-rise apartment building which has five or more storeys. Also included are apartments in a building that has five or more storeys where the first floor and/or second floor are commercial establishments.

6. Apartment in a building that has fewer than five storeys

A dwelling unit attached to other dwelling units, commercial units, or other non-residential space in a building that has fewer than five storeys.

7. Other single-attached house

A single dwelling that is attached to another building and that does not fall into any of the other categories, such as a single dwelling attached to a non-residential structure (e.g., store or church) or occasionally to another residential structure (e.g., apartment building).

8. Mobile home

A single dwelling, designed and constructed to be transported on its own chassis and capable of being moved to a new location on short notice. It may be placed temporarily on a foundation pad and may be covered by a skirt.

A mobile home must meet the following two conditions:

• It is designed and constructed to be transported on its base frame (or chassis) in one piece.
• The dwelling can be moved on short notice. This dwelling can be easily relocated to a new location, because of the nature of its construction, by disconnecting it from services, attaching it to a standard wheel assembly and moving it without resorting to significant renovations and reconstructions.

9. Other movable dwelling

A single dwelling, other than a mobile home, used as a place of residence, but capable of being moved on short notice, such as a tent, recreational vehicle, travel trailer, houseboat or floating home.

Private dwelling type codes — Chart

Is this dwelling attached to another dwelling or structure (other than its own garage or shed)?

• No
  Can this dwelling be moved on short notice?
  • No: Code 1
  • Yes
    Is this dwelling designed and constructed to be transported on its own frame (i.e., mobile home)?
    • No: Code 9
    • Yes: Code 8
• Yes
  Does this dwelling have any other dwelling(s) above or below it?
  • No
    Is this dwelling in a building that has more than two dwellings attached side by side or back to back?
    • No
      Is this dwelling attached to only one other dwelling side by side or back to back (i.e., semi-detached)?
      • No: Code 7
      • Yes: Code 2
    • Yes: Code 3
  • Yes
    Is this dwelling in a building that has five or more storeys?
    • No
      Are there exactly two dwellings in this building?
      • No: Code 6
      • Yes: Code 4
    • Yes: Code 5