About the IS4H-MM Assessment

LEVEL 1

• Data is not collected, and some data is available from external estimations (international organizations)

Data Sources

• Indicators are not generally produced by the national health authorities.
• Indicators rely heavily on estimates from international organizations.

Information Products

• There are few, if any, formal data standards enforced.

Standards for Quality and Interoperability

• There a few if any best practices for data management implemented.
• Data management is largely ad hoc.
• There are no formal mechanisms for decisions about data quality and standards.

Data Governance

• Basic tools and technology (hardware, software, internet connectivity) are not widely available.

IT Infrastructure

LEVEL 2

• Data is sometimes obtained from few sources.
• Data frequently has limited utility because of quality or disaggregation issues.
• Data are largely collected using paper-based methods, although a few simple electronic tools like spreadsheets may be used for some data sources routinely collected electronically from all key sources.
• Some indicators definitions are defined but not easily accessible/shared.

Data Sources

• Some information products are generated, but not routinely, require intensive work and use of resources.
• Data is not readily shared across units, with stakeholders or public.
• Sharing data frequently requires permission from senior levels.
• Indicators generation largely relies on data from survey's, census, and other ad hoc studies.

Information Products

• Some standards are defined in individual data sources but are not consistent or available across data sources.
• Standards for interoperability have been identified but not implemented.

Standards for Quality and Interoperability

• Data management best practices are in development, but not fully implemented.
• Data quality is not routinely monitored.
• Decision about standards quality and standards are made at the facility/unit/team level.

Data Governance

• Basic Tools are generally available but may be older or not performing well.

IT Infrastructure

LEVEL 3

• Health data are routinely collected from key data sources.
• Data is collected electronically using a variety of tools like spreadsheets, databases and electronic client-based information systems.
• Integration from different sources is often a manual process, and may be constrained by comparability issues.

Data Sources

• A range information product are efficiently and routinely produced from country information systems.
• Dissemination of information products is typically limited to senior-level decision makers.

Information Products

• Some standards have been identified for specific data sources, and there are formal plans for adoption.

Standards for Quality and Interoperability

• Data management processes and best practices are implemented for some facilities/units/teams. (e.g. data quality frameworks, data standards, policies, SOPs.)
• Core data sets are readily available.
• Data are often integrated for analysis across various sources.
• Some metadata are documented and maintained (indicator compendium, data dictionaries) support by a data governance body within the national health authority.

Data Governance

• There is evidence of interoperability between some health information platforms.

IT Infrastructure

LEVEL 4

• Health data is derived routinely and timely form all key data sources.
• The country carries out proactive activities to improve data collection processes.
• Some data is available in near-real time to support decision-making.

Data Sources

• Formal health data governance mechanisms have been established at the national level with other health data stakeholders.
• There are processes and plan in place to strengthen alignment of standards, data quality frameworks and data management practices across all stakeholders.

Data Governance

• A national identifier is available for integrating health data from all sources.
• Standards have formally adopted, and the national health information architecture has been documented.

Standards for Quality and Interoperability

• Information products are routinely produced that meet the specific needs of various stakeholders and are routinely distributed to stakeholders at all levels of the health system.

Information Products

• Widely available and interoperable across the public health system.

IT Infrastructure

LEVEL 5

• Large data sets integrated from multiple sources are readily available for analysis to support decision making.
• Data from multiple data source types, including unstructured sources such as social media and various types of devices are used in health analysis.
• Large data sets integrated from multiple sources are readily available for analysis to support decision-making.

Data Sources

• Data management policies, procedures and best practices are consistently applied, resulting in availability of quality data.
• Formal data governance mechanisms (committees, policies, data quality frameworks, data sharing agreements, etc.) have been established among national health stakeholders and are effectively functioning.
• Continuous improvement processes established to monitor and invest in data quality.

Data Governance

• Information systems for health are interoperable, enabled by a national infrastructure that uses current standards, technologies, and architectures.

Standards for Quality and Interoperability

• Information products are developed from a range of structured and unstructured data sources. Data for decision-making is available in near real time to all stakeholders.

Information Products

• There is evidence of significant interoperability across health platforms.
• Integrated national repositories from multiple data sources.

IT Infrastructure

LEVEL 1

• Accountability and decision-making for IS4H is distributed across different units within national health authorities, and investments and activities are typically not coordinated.

Leadership and Coordination

• There is no current National Health System Strategic Plan, and IS4H components are not reflected in operational plans.

Strategic and Operational Plans

• Some IS4H functions are formally defined and performed, but there are significant gaps.

Organizational Structures and Functions

• There is little awareness of the human resource requirements to support IS4H.

Human Resources

• IS4H activities/resources are not formally identified in program/unit budgets.
• While it is sometimes possible to secure one-time financial resources for IS4H investments, required investments are difficult to sustain.

Financial Resources

• Identified key stakeholders are from the public health sector exclusively.

Multisectoral Collaboration

• There is general awareness that there are gaps in legislation, policy and compliance mechanisms that create barriers to the effective use of IS4H, but specific gaps and needs have not been formally documented.

Legislation Policy and Compliance

• There is some awareness of data and reporting obligations under national and international agreements, but little capacity to meet obligations.

National and International Agreements

LEVEL 2

• IS4H investment decisions are coordinated at the management level within individual national health authorities (e.g., MOH, reginal health authorities, health facilities, etc.) but not formally coordinated among health authorities or other national actors.

Leadership and Coordination

• There is a National Health System Strategic Plan, but it does not address IS4H. Some individual units/departments/facilities include some components of IS4H in their operational plans.

Strategic and Operational Plans

• There are gaps in IS4H services or functions, and/or services and functions may be duplicated across units/programs.

Organizational Structures and Functions

• There are identified human resource constraints for planning, implementing, and managing IS4H, but there is no formal plan for addressing human resource needs.

Human Resources

• IS4H activities are identified in individual unit/program annual budgets of national health authorities, but are not integrated or aligned across units/programs.
• The financial resources requirements to effectively sustain IS4H have been identified, but not fully secured within operational budgets.

Financial Resources

• There are some relationships with other public sector stakeholder for specific some information and service needs. However, engagement and coordination are ad hoc.

Multisectoral Collaboration

• Requirements for IS4H enabling legislation, policy and compliance mechanism have been identified, but not yet implemented.

Legislation Policy and Compliance

• Data and reporting obligations under national and international agreements are frequently met, but with high resource impact.

National and International Agreements

LEVEL 3

• There is a formal governance structure in place for strategic planning and oversight of IS4H among the national health authorities (e.g. MOH, regional health authorities, health facilities, etc.).

Leadership and Coordination

• There is a current National Health System Strategic Plan that include priorities for strengthening health information.
• IS4H is included within operational plans of national health authorities.

Strategic and Operational Plans

• Accountability and responsibility for IS4H functions within national health authorities have been defined
• There are plans in place for organizational restructuring or re-alignment to rationalize functions and decision-making.

Organizational Structures and Functions

• There are policies and SOPs that address ethical use and protection of health data (e.g., privacy, security, secondary use), but there may be gaps in regulation or legislation.

Legislation Policy and Compliance

• Skills and job functions required to effectively support IS4H have been identified, although not all resources have yet been secured.
• There are is some evidence of competency building activities (training, workshops, conferences) for IS4H domains but these are typically ad hoc.

Human Resources

• There is a plan in place for resource mobilization for specific IS4H capital investments.
• Financial resources secured for the sustainable implementation.
• Operations of IS4H have been secured with annual budgets.

Financial Resources

• Informal relationships have been established with key multisectoral national actors, including private sector organizations.

Multisectoral Collaboration

• Data and reporting obligations under national and international agreements are consistently met with an effective use of resources.

National and International Agreements

LEVEL 4

• IS4H governance structures are established at the national level across at least some key national stakeholders (e.g. health authorities, national IT authorities, vital statistics, national statistics authorities).

Leadership and Coordination

• An organizational structure that defines clear accountabilities and responsibilities for IS4H has been fully implemented within/among national health authorities and is reflected in unit/program mandates and job descriptions.

Organizational Structures and Functions

• Sufficient human resources with the required skills to effectively implement and sustain IS4H have been secured.
• Relevant IS4H skills and competency development are integrated into training plans for leadership, management and staff.

Human Resources

• There is a formal strategic plan in place among national health authorities for strengthening IS4H that reflects the IS4H Strategic Framework.
• Operational plans of the units within national health authorities reflect IS4H activities and outcomes based on the IS4H Strategic Plan.

Strategic and Operational Plans

• An IS4H investment framework is established at the national level.

Financial Resources

• Formal relationships have been established with multisectoral actors, including the private sector. There are examples are collaborative initiatives between multisector partners.

Multisectoral Collaboration

• Agreements enable data and information sharing across national and international stakeholders.

National and International Agreements

• The legislation, policies, and compliance mechanism required to effectively implement and operate IS4H are fully implemented.

Legislation Policy and Compliance

LEVEL 5

• The governance and management of IS4H is fully transparent and integrated across national stakeholder organizations.

Leadership and Coordination

• IS4H is fully sustainable, supported by an investment model that ensures the required human resources, processes, legal-ethical framework, knowledge and technologies are available to deliver ISH effectively, and to continually invest in new capabilities as they emerge.

Financial Resources

• The legal-ethical framework fully enables the use of information and technology to improve health outcomes and the performance of the health system while protecting individuals and populations and is responsive to emerging innovations.

Legislation Policy and Compliance

• IS4H functions are defined, performed and aligned across national stakeholders. Functions are integrated across national stakeholders, optimizing performance, value and investments at the national level.

Organizational Structures and Functions

• There is a national strategy for building IS4H human resource competencies that includes national and international educational institutions to ensure the long-term availability of skilled IS4H resources.

Human Resources

• There is a National IS4H Strategic Plan.
• Operational plans are aligned and integrated across multisectoral stakeholders.

Strategic and Operational Plans

• IS4H governance includes representation from multi-sectoral partners.
• IS4H roles, responsibilities and functions are aligned across multisectoral partners.

Multisectoral Collaboration

• Data and information can flow freely among national and international partners in support of agreements, guided by frameworks that ensure the ethical use of information that protects individuals and populations.

National and International Agreements

LEVEL 1

• Knowledge sharing in the organization is ad hoc and Organizational knowledge resides with key individuals rather that on repeatable processes documented in unit descriptions, job descriptions, policies and SOPs.

Knowledge Processes

• Knowledge management is felt as a need, but there is little knowledge and expertise in this matter.
• Although some basic knowledge management technologies and tools are available (physical library of internal resources, shared drives), they are not consistently or organized.
• Accessing organizational knowledge is time-consuming and difficult.

Knowledge Architecture

• There are routine public health communications on national priority issues (e.g., healthy lifestyle, vector control, etc.).
• Data and information typically flow only from source to the central level.

Strategic Communications

• Communication with civil society and the public is typically “one-way” (e.g., through websites and advertising).

Social Participation

• No formal relationships have been established between health authorities and the academic/scientific community.

Academia/Scientific Community

• Networks for knowledge sharing are typically ad hoc and informal.

Networks

LEVEL 2

• There are some basic knowledge management mechanism and processes (e.g. formal meeting notes, trip reports, SOPs, documentation etc.) in place randomly accessible and updated, but these are not required in policy or practice.

Knowledge Processes

• There is an awareness among leadership and staff of the key concepts and importance of knowledge management.
• Some isolated KM&S pilot projects (not necessarily by management initiative).

Knowledge Architecture

• An informal public health communication strategy in in place, not operationalized.
• Public health strategic communications include healthy lifestyle and prevention issues.

Strategic Communications

• There is limited engagement with civil society and the public through basic mechanisms such as surveys and focus groups.

Social Participation

• Relations with academia are fluid, informal and on demand.

Academia/Scientific Community

• Staff participate in knowledge networks (e.g. communities of practice, conferences, listservs) on ad hoc basis.

Networks

LEVEL 3

• There are numerous knowledge management processes defined (lessons learned, trip reports, mentoring, shadowing, etc.) guided by formal policies and procedures.
• There is a formal basic KM strategy at the organizational level.

Knowledge Processes

• Basic KM infrastructure (e.g., share information repositories, content management standards, etc.) is in place.
• Some CG projects have been launched at some levels of the organizational structure.
• KM skills strengthening is part of a training program.

Knowledge Architecture

• A formal public health communication strategy in place with targeted messages to specific audiences.

Strategic Communications

• The participation of civil society in the health system is actively encouraged through social media and formal roles on governance bodies and advisory groups.

Social Participation

• Formal relationships with academia have been established to expand organizational knowledge and learning.

Academia/Scientific Community

• Participation in communities of practice is encouraged and staff routinely capture and share knowledge from these forums.

Networks

LEVEL 4

• Knowledge management and sharing is integrated into business processes, job descriptions and organizational functions.
• Metrics are used to quantitatively measure organizational knowledge management processes and capacities, and continuously improve performance.

Knowledge Processes

• A formal knowledge management framework has been established within the organizations, with robust policies, processes and mechanisms for knowledge management and knowledge sharing.

Knowledge Architecture

• Civil society organizations and the public are constantly engaged.

Social Participation

• National authorities can measure the impact of strategic communications and adjust communications strategies accordingly.
• There is a public health communication strategy with defined messages customized for specific audiences and purposes informed by national evidence.

Strategic Communications

• Formal relationships have been established with academia/scientific community focused on supporting projects and programs with specific studies.

Academia and scientific community

• Knowledge networks are integrated into organizational structures and practices by a resources and compensations program.

Networks

LEVEL 5

• Health authorities and their multisectoral partners are fully learning organizations.
• The organizational culture encourages the free-flow of knowledge throughout the organization, enabled by KM processes, tools and technology.

Knowledge Process

• The KM&S systems are fully operational. Integration of technology with content architecture.

Knowledge Architecture

• Strategic communications are informed by advanced analytics in near real-time.

Strategic Communications

• Decisions by health authorities and other health system actors are transparent, driven by evidence and engagement with civil society and the public.

Social Participation

• Formal relationships have been established with academia/scientific community focused on supporting specific projects or studies, support decision-making and programs evaluation.

Academia and scientific community

• As an integrated organizational practice, participating and creating networks is focused in helping the organization to continually identify and adopt emerging knowledge.

Networks

LEVEL 1

• Leadership and staff are not familiar with IS4H concepts.

Key Concepts

• Standard statistical analysis is routinely applied to available health data to generate reports on health status and outcomes.
• Most health analysis is focused on the generation of indicators, although other types of health analysis are done on an ad hoc basis are required for special presentations and projects.
• Information is used to support decision-making in limited circumstances, but evidence-informed decision making is integrated into the policy and management culture.

Health Analysis for Decision-making

• Basic tools are routinely used for health analysis (e.g., spreadsheets, MS Access, etc.).

Tools

• Health care delivery and services are largely manual processes.
• Assessing digital technologies in health incl health information systems at national/subnational level to identify areas of improvement.

Digital Health

• E-government is not on the national agenda.

eGovernment

• The concepts of Open Government are new to leadership.

Open Government

• Manual and electronic health information systems are vulnerable to failure in the event of a natural disaster or other catastrophic event.
• Limited data available to support disaster response .

Preparedness and Resilience

LEVEL 2

• While some IS4H concepts are understood, leadership and staff are not widely aware of all concepts.

Key Concepts

• Basic tools are routinely used for health analysis (e.g., spreadsheets, statistical packages, etc.) and data is stored in relational databases.

Tools

• Data typically flows from sources to central decision-makers for health analysis, but little health information is available for decision-making at the local level.
• There is evidence that data and information are routinely used to support policy and management decision-making.

Health Analysis for Decision-making

• Digital health tools such as electronic records, laboratory/pharmacy information systems and electronic order entry are being implemented with a focus on digitizing manual processes and operational efficiencies.
• Developed roadmap based on assessment to better integrate digital technologies into existing health systems including normative and technical aspects.

Digital Health

• E-government is on the national agenda, but there is no formal strategy or unit in place.

eGovernment

• There is broad knowledge of open government principles among national health authorities, and leadership support for advancing open government policies and initiatives.

Open Government

• There is evidence of approaches for ensuring business continuity in the case of disaster (e.g., routine off-site backups, downtime manual process SOPs, etc.).
• Some key data sets are available to support disaster response (e.g., facilities and health human resource databases, database of emergency centres, mortality data, etc.)

Preparedness and Reliance

LEVEL 3

• Most leadership and staff have an understanding of IS4H concepts. There are recent assessments that demonstrate strong digital literacy among most leadership and staff.

Key Concepts

• All essential information to support clinical, management, policy decision-making and is readily accessible, and end-users have on-demand access to information products or health analysis resources.
• There is capability among clinicians, administrators, and policymakers for evidence-informed decision-making, and clinical, management and policy decisions are data-driven.
• A range of health analysis approaches are routinely applied (e.g., ASIS ARMAR7, Health Inequalities, Multiple Cause of Death, etc.)

Health Analysis for Decision-making

• Advanced tools are routinely used for health analysis (e.g., spreadsheets, statistical packages, etc.) and all data is stored in relational databases.

Tools

• There is evidence of digital health tools being used to transform models of care, improve patient safety and quality of care, or for supporting population health approaches.
• Appropriate legislation and data protection policies around data access, sharing, consent, security, privacy, interoperability are being developed.

Digital Health

• The government has established an e-government strategy or unit.
• Currently the focus is on strengthening core IT infrastructure.
• Health is not a core stakeholder.

eGovernment

• Open data principles have been formally adopted in policy.

Open Government

• There is evidence that health information systems would be resilient during disasters and are able to able to support essential health system functions and disaster response.

Preparedness and Resilience

LEVEL 4

• There is advanced capacity among technical staff. Continuous capacity building (investment in skills, tools, partnerships) for more advanced approaches of health analysis.

Health Analysis for Decision-making

• Advanced tools are routinely used for health analysis (e.g., spreadsheets, statistical packages, etc.) and all data is stored in relational databases and new approaches for non-traditional databases are initiated., tools are continuously updated and improved.
• Online data platform is available.

Tools

• Knowledge of IS4H Key Concepts and digital literacy is high among leadership and staff, and there is evidence that these concepts are routinely applied in practice at all levels of the organization.

Key Concepts

• Digital health tools are used to facilitate targeted communications to individuals to stimulate demand for services/access to health information.
• Digital health interventions are targeted to health workers to give them immediate access to improve decision support mechanisms / telemedicine.

Digital Health

• Open data principles are fully applied, and key data sets are available for analysis by other national and international stakeholders.

Open Government

• Health information systems would be resilient during disasters and are able to able to support essential health system functions and disaster response.

Preparedness and Resilience

• There is evidence of eGovernment initiatives that are transforming transactions between government and the public, businesses, or other organizations in health (e.g. online appointment booking, patient portals, e-referral, health card registration, etc.)

eGovernment

LEVEL 5

• There is expert knowledge and capacity among technical staff that go beyond routine analysis required. There is annual capacitation and budget towards training.
• Health Analysis can be done real-time and routine clinical, management and policy decision-making are based on timely analysis.
• Data driven decision-making for public health strategies and activities.

Health Analysis for Decision-making

• Online tools and platforms for data dissemination and analysis (e.g., data repositories, dashboards, portals, visualization tools, spatial data, etc.) are appropriately and securely available for different user types, such as policy makers, manager, clinicians, and public stakeholders.

Tools

• Knowledge of IS4H Key Concepts and digital literacy is high among leadership and staff, and there is evidence that these concepts are routinely applied in practice at all levels and across sectors.

Key Concepts

• Open data principles are fully applied. Full interaction with national and international partners regarding the use of data analysis to strengthen decision making.

Open Government

• IS4H are fully resilient during disasters. The operation of information systems for health and access to information is available during and after emergencies and disasters.

Preparedness and Resilience

• The health sector is fully integrated into e-government initiatives and platforms.

eGovernment

• Digital health technology enables population health management and the rapid response to disease incidents and public health emergencies. Citizens are empowered to manage their own health and to proactively engage with health care providers. Health care workers have access to data and tools that support real-time decision making.

Digital Health