Toolkit

The template for  organizational context tabulates the factors indicated as relevant at ISO 21001 for the identification of the internal context: values, culture, knowledge and performance; and external context: legal, technological, competition, commercial, cultural, social and economical. The table is organized in such a way to allow this identification to be done by process, instead of for the organization as a whole. The advantage of doing the identification of the internal and external context by process is that it facilitates the identification of more detailed aspects that might be relevant for a given process, but not for the whole organization, and that might be missed otherwise.

Users should start by identifying their processes on column A and then, having each process in mind, proceed to:

1. use columns B to E to categorize each internal context aspect identified and column F to describe it;
2. use columns G to M to categorize each external context aspect identified and column N to describe it;

Rows should be added to the table as necessary to enable the inclusion of all aspects identified as relevant.

This tool in editable format

An example of the use of this tool

In the management systems world, since the introduction of the high-level requirement on organizational context, common to all ISO management system standards, it became a trend for organizations to use a SWOT Analysis to identify their internal and external context.

Although this seems to be a straightforward approach, it might not facilitate a deeper analysis, considering all factors – particularly external factors – listed at ISO management systems standards as potentially relevant.

To overcome this risk, a PESTEL-SWOT combined analysis can be performed, also onboarding some internal factors mentioned in standards such as ISO 31000 on risk and ISO 22301 on business continuity.

The template to make such combined analysis has the four areas of the SWOT analyses, but also lists the relevant factors to consider, thus helping deepen the reflection.

This tool in editable format

An example of the use of this tool

Another example of the use of this tool

The template for principles, policies and objectives tabulates the EQAVET and ISO 21001 requirements of the respective clauses, making it straightforward for educational organizations to deploy a strategy in an objective, systematic and articulated way.

Once the applicable principles are inserted in the template, policies derived from these can be established/adopted, and SMART objectives to deploy them can be determined for any given period. This includes all necessary details to plan the achievement of objectives, such as actions, resources, responsibilities, deadlines and methods to monitor and measure its achievement.

This tool in editable format

An example of the use of this tool

Another example of the use of this tool

This tool aims to define the measures or actions to be implemented by the educational organization over a medium to long-term period of time.

The educational organization should start by conducting a self-reflective review a diagnosis starting with an analysis of the general and sectoral external environment, including the reflection on the threats, the opportunities and the critical factors for success.

Such a reflection should be followed by an internal diagnosis, in which the resources (premises, equipment, human resources, etc.) are considered, as well as the strengths and weaknesses.

This tool shall also be used to identify the Mission, the Vision, the Values, and the strategic objectives for the period under review.

Finally, the educational organization should establish a planning matrix which will provide the connection between the strategic objectives and the corresponding measures or actions to implement them. These will require the identification of those responsible, the implementation schedule, the performance indicators as well as any monitoring and measurement.

This tool in editable format

An example of the use of this tool

The EO’s Annual Activity Plan considers the recommendations inserted in the Annual Report.

The document is essential for the EQAVET implementation methodology since it provides us with an updated plan.

The document targets all internal and external interested parties.

This document is related to all the other documents that are referred to the EO’ activity, namely the Action Plan.

This document relates to ISO 21001 Clause:6.2 as it is an essential document for the planning of changes.

It is related to all the other documents that are referred to it.

This document also relates to ISO 21001 Clause:6.1.1, as it specifically has several items concerning the revision of activities, the institutional dialogue for the continuous improvement of the VET offer and the application of the continuous improvement cycle.

This tool in editable format

An example of the use of this tool

The template for processes follows the process description approach recommended by ISO 21001. In this approach, a process is defined as a set of interrelated activities that transform inputs into outputs/results by adding value to them. This approach also considers an internal supplier/internal customer philosophy. To reflect this, the template has columns to describe the activities in each process, the inputs necessary to perform those activities (and where they come from) and the outputs expected as results (and to whom are they destined to). Complementary to the ISO 21001 vision, this VET21001 tool adds a column to the template to identify who is responsible to perform each activity. Although it might be counter intuitive, the first columns to be filled in this template are the central ones with the description of tasks and who is responsible for it. After those columns are filled, which can be done in text or with a flowchart, it is much easier to determine which inputs are needed to perform them and who is the internal or external supplier who will provide them. Likewise, it facilitates the identification of the expected outputs, which can be intermediary or final, and to which internal or external customer are they destined and should be handled to.

This tool in editable format

An example of the use of this tool

This Action Plan (AP) defines the implementation of surveys on students, parents, teacher/trainers and other collaborators. This action plan defines the activities, inputs, responsible and or organs and others involved as well as the execution mode, deadlines, expected results and modes of dissemination. The document is essential for the EQAVET implementation methodology since it provides the EO (educational organization) with an action plan for receiving the contributions of learners and other beneficiaries.

The document targets all internal interested parties.

This document is related to all the other documents that are referred to the EO activity, namely the intermediate and annual reports.

It relates to ISO 21001 Clause 5.1.2, as it also addresses learners and other beneficiaries.

On the other hand, the Action Plan determines processes for the revision of activities, the institutional dialogue for the continuous improvement of the VET offer and the application of the continuous improvement cycle. With these activities, the EO determines the risks and opportunities that need to be addressed to give assurance that it can achieve their intended outcomes, enhance desirable effects, prevent, mitigate or reduce undesired effects and achieve continual improvement.

The document is essential for the EQAVET implementation methodology and is updated whenever it is necessary, according to the organization’s needs or the public authority orientations.

The document targets the whole EO interested parties, namely the founders of the organization, companies of the sectors served by the school, as well as internal staff and other providers. But it is, essentially, an internal document, for the support of EQAVET implementation.

This tool in editable format

An example of the use of this tool

The template for risk-actions tabulates the ISO definition of risk and its columns are divided in two main sections:

1. the description of the risks
2. the description of the actions to address the risks

The first section contains fields to describe the deviation from the expected that can arise from the state of deficiency of information related to the understanding, or knowledge of an event, its consequences or likelihood – in other words, to describe the effect of uncertainty, which is the ISO 21001 definition of risk. It also contains fields to describe those potential events and consequences, as well as it’s likelihood of occurrence; and the source of the risk, when this is known.

The second section contains field to record the decisions regarding the acceptability of the risks, as well as to categorize and describe the actions taken or to be taken to address the risk.

The intent of this table is to facilitate deeper, evidence-based, reflections about risks and boost a systematic approach on how to address them.

This tool in editable format

An example of the use of this tool

According to ISO 31000, risk is the “effect of uncertainty on objectives” and an effect is a positive or negative deviation from what is expected. Therefore, the risk analysis should analyse both threats and opportunities.

Risk analysis is a process that is used to understand the nature, sources, and causes of the risks that you have identified and to estimate the level of risk. It is also used to study impacts and consequences and to examine the controls that currently exist. How detailed your risk analysis ought to be will depend upon the risk, the purpose of the analysis, the information you have, and the resources available.

Risk can be identified during a swot analysis, an audit, a top management review, during the normal processes of the educational organization, etc. The “Template Risk Matrix” identifies first the origin of the risk. The educational organization should identify any risks to processes, organization, Reaching the goals, impartiality, lack of resources (material and human), etc. Should be considered, as well as, internal and external factors that can, in any way, be a risk (threat or opportunity).

The Risk matrix is a simple tool that allows organization to identify the risk, classify it and define the measures according to the risk classification. It also allows organizations to register the control (evidence/efficiency) of each measure.

This tool in editable format

An example of the use of this tool

Another example of the use of this tool

The template for indicators serves to define indicators to monitor the performance of the quality management system processes, based on the aim of each process and the planed targets to reach. It also contains fields to define the monitoring method in which the indicators will be used – in other words, how the indicators will be used – as well as the periodicity of its application and who is responsible for doing so. Besides serving as a monitoring plan, the template can also be used to support records, as it includes fields to register the results obtained and any comments to those, as well as the evidence in which the results are based and its location, to assure full traceability.

This tool in editable format

An example of the use of this tool

The template for integrated improvement system, also known as the SIM table, is a master management tool, which allows for a systematic approach to early identification and analyses of problems and potential problems, as well as evidence-based decisions on how to act on them and how to follow-up those action to assure its effectiveness.

The table is divided in four main sections, containing a set of columns to:

1. identify the situations
2. analyze the situations
3. define actions to address the situations
4. define methods to verify the implementation of the actions defined
5. define methods to verify the effectiveness of the actions implemented

The tool was also designed to serve both as a supporting document (with columns’ titles that are self-explanatory and provide users with guidance on how to handle situations) and as a record (by providing fields to insert information that will allow for future traceability of situations and actions taken to address them). Across time, the information inserted in this tool constitutes a good repository for organizational knowledge based on lessons learned. Any organization who effectively implements this tool and uses it realistically will be very well positioned to master continuous improvement in the long term.

This tool in editable format

An example of the use of this tool

1.9. Measures are designed to ensure compliance with data protection rules

This template was developed in order to control Internal supporting Documents, External supporting documents and Records.

It starts by identifying the process to which the document it’s allocated. The several characteristics of the document, such as, Identification, description, format, the person /job role responsible for the review and approval, version, access, distribution, location, access protection, preservation, retention and disposition.

The obsolete documents and control should also be included in the document control.

This tool in editable format

An example of the use of this tool

Another example of the use of this tool

Document Control is a requirement of most Quality Assurance Standards, including ISO 21001. The Document Master list was created with the intention of providing Institutions with a tool to easily manage such a requirement. The template is formatted in such a way to enable users to record the important details pertaining to every document forming part of their Quality Management System (QMS).

A document control list/tool can take many forms. This particular version provides users with a selection of  minimum data/information which is to be recorded as part of this process. Although the form provides this particular selection, it does not preclude users from adding fuirther information and/or details according the needs of their organisation. It is highly recommended however, that the minimum sections provided as part of the template are kept and duly recorded.

Apart from the recording of individual document data, the tool also includes a DCN Register (sheet 2). DCN stands for Document Change Number, and as the name suggests, this register enables users to keep track of the individual changes applied to a given document. In some cases, the same document may be updated multiple times throughout the course of an academic year. Each change would update the revision of the document, thus creating a situation where various revisions are released, potentially making users lose track/focus of these changes. By registering the individual changes via the DCN Register, users would be able to track these individual changes, thus forming an audit trail providing further visibility on the development of a given document.

This tool in editable format

An example of the use of this tool

In training services, the acquisition of personal data of users/learners is constant. The goal of the privacy policy statement is to inform them about how their data is being processed. It should be accessible to your users and formulated in clear and simple language, providing the relevant information about what personal data you are acquiring and processing, for what purpose, how you disclose and disseminate it to internal and external parties, what rights the user can assert, who are responsible, and how you protect it. The following template is intended to provide a simple basis for developing a privacy policy statement in situations where services or activities performed by the organization require the acquisition of data processing consent from the user.

This tool in editable format

An example of the use of this tool

This tool in editable format

An example of the use of this tool

Competence applies to any work that directly or indirectly affects organizations’ ability to consistently deliver products and services that meet applicable legal and regulatory requirements and improve customer satisfaction. A job description is used to determine the competences required for each organizational role to achieve a job performance that will ensure expected results in line with the organization’s performance objectives. These competences can be determined for a single group or for groups of similar jobs with each other. Beware because this need is not fixed but can change over time and should therefore be reviewed periodically and not only when a new job position is implemented within the organization (i.e. when there are new job specifications, new operating instructions, new technologies, when plans are made for the adoption of new equipment, when an analysis of nonconformities, customer complaints and other issues dictates that new skills must be sought to improve the situation).

Determining necessary competence entails identifying the required results or outcomes of a job, position, task or role, the criteria or standards of performance to be achieved, the evidence required and the method for achieving them (refer to table for processes). The starting point, then, is to define the outcomes expected from a job in order to define, consistently, what makes those who perform it successful practitioners and to establish the basis for competence assessment and development.

Organizations beginning to work on this could start with a risk-based approach, determining the potential impact on product and service compliance and customer satisfaction of not applying the competence requirements in certain cases. This requires the organization to determine and map the competences needed by the person(s) whose work falls within the scope of the standard and which affects the efficiency and effectiveness of the management system, and to ensure that those individuals are competent by education, training or experience.

Establishing the necessary competencies for all job roles helps focus the minds of managers and employees on what criteria are important for achieving certain performance within a particular job role. This helps significantly during the human resource recruitment and growth processes of the organization, as well as in identifying skills mismatches and professional development opportunities for the staff coherently with individual job roles and organization’s objectives. Where there are mismatches between the competencies needed and those possessed by personnel, action should be taken to enable them to acquire the necessary competencies (see Training plan).

This tool in editable format

An example of the use of this tool

The template covers the relevant information to achieve the benefits of structured competency management:

i) training needs analysis to identify gaps in individuals’ knowledge and skills with respect to their job role and the changes and outcomes the organization intends to achieve;

ii) planning and scheduling of necessary development and training activities consistent with identified training needs and objectives;

iii) implementation and monitoring of training interventions, including costs, status, duration, training providers, and type and area of training; and

iv) follow-up of each individual’s progress and retention of related documented information.

This tool in editable format (Word)

This tool in editable format (Excel)

The template for interested parties is based on the definition of interested party/stakeholder given by ISO 21001 and therefore has columns to categorize each interested party identified as:

1. being internal or external to the educational organization; and
2. affecting, being affected or perceiving itself as being affected by the educational organization.

Other columns exist to identity the interested parties and to describe the way it affects, is affected or perceives itself to be affected by the educational organization.

The table is organized in such a way to allow the identification of interested parties to be done by process, instead of for the organization as a whole. The advantage of doing this identification by process is that it facilitates the identification of more interested parties that might be relevant for a given process, but not for the whole organization, and which might be missed otherwise.

The template provides a structured and systematic approach for identifying interested parties.

This tool in editable format

An example of the use of this tool

Another example of the use of this tool

The template to control external providers tabulates the detailed requirements contained in ISO 21001 – as these are more detailed than EQAVET ‘s – to identify, select and evaluate/re-evaluate external providers, as well as to verify the fitness for purpose of the products/services they provide so a decision regarding its acceptance can be made.

The template contains fields to first organize the products and services provided by external providers in categories, as typically, external providers of the same “family” which provide products and services of the same nature, share criteria. Therefore, starting by doing this organization enables a more efficient evaluation process, as criteria can be determined per category, saving substantial time and effort, particularly to educational organizations that depend on many external providers.

The template also contains fields to identify the associated process of the management system or, in other words, the process that will receive the externally provided product or service – which will also be identified in the template – as an input.

Depending on the nature of the product or service externally provided, there might be a risk of counterfeiting and this should also be identified in the template, together with the measures deployed to control this risk. Also depending of the nature of the product or service externally provided, its impact in the quality of the products and services provided by the educational organization to their learners and other beneficiaries varies and should be identified in the template as high, low or negligible impact. High impact requires the definition of stronger criteria to control the external providers and the products and services they provide than low impact. Negligible impact doesn’t justify the effort necessary to implement systematic control external and these can therefore be skipped.

The template proceeds with fields to identify the external provider under evaluation and the criteria defined to select them and to periodically evaluate their performance after being selected. It also contains fields to define the criteria needed to verify the products and services when they are delivered and to decide on its acceptance.

The template finalizes with a section to identify the qualification status of each external provider and to add any relevant comments to that status

This tool in editable format

An example of the use of this tool

Another example of the use of this tool

Supplier evaluation is a term used by organizations to evaluate and approve their existing and potential suppliers through a series of assessments. This evaluation consists of a series of questions based on parameters like competency, capacity, consistency, quality, etc., with a view to assessing the suppliers to ensure an approved portfolio via the application of Supplier Evaluation Tools.

Evaluating suppliers can be challenging, hence, the best practice is to create an evaluation form that will help organize and evaluate the suppliers more efficiently. This evaluation form is essential in order to monitor and measure the supplier’s performance to reduce the cost associated with the activities, analyze risk management and maintain scope for constant improvement in selecting the suppliers.

The Carter 10Cs model that is recognized worldwide, and on which this tool is based, looks into the aspects of evaluating the supplier before being appointed. The 10 Cs are listed below:

1. Capacity – Does the supplier have the bandwidth to deliver?
2. Competency – Is the supplier diligent and can complete the task in a given period of time?
3. Consistency – Is there a consistent output from the supplier?
4. Control of process – Does the supplier offer flexibility and have systematic control over process?
5. Commitment to Quality – Is there a system established by the supplier that works constantly for quality management checks?
6. Cash – Is the supplier financially independent or is there third party involvement?
7. Cost – Are the products and services offered by the supplier cost-effective?
8. Culture – Does the supplier have a good work culture and market reputation?
9. Clean – Does the supplier have a legitimate license, does not favor uncertified professionals and is risk management certified?
10. Communication efficiency – Does the supplier have the latest means of communication to answer queries?

Based on these important parameters, the supplier organization can be asked to furnish all the details right after the screening process is completed.

The supplier Evaluation Form will equip any organization with the capacity to easily identify the correct suppliers for a specific need. Evaluating suppliers will provide organizations with the peace of mind that they are ordering from a reputable source, and that therefore risks will be minimized. Moreover, the records and assessments done for suppliers following each service, will provide a clear picture of the supplier’s consistency, thus providing organizations with the information needed on whether the supplier can be considered as an Approved Supplier. The questions asked in the evaluation form help organizations come to a solid conclusion whether or not the supplier should be appointed or re-appointed in the future.

Other benefits of having an evaluation form ready for supplier evaluation include:

1. 1. Helps in assessing any risk involved: If an organization is not aware of the supplier, getting an assessment done prior to ordering will help the organization know their supplier and they can assess any risk associated with appointing them. Moreover, if the same supplier is appointed multiple times, and their evaluation is always positive, it will grand further confidence to the organization that the risks are mitigated even further. As Carter 10Cs model suggests, risk management can be associated with capacity, competence and commitment to quality.
2. Helps in protecting the brand reputation: Through these evaluation forms, organizations can appoint suppliers with a good market reputation and, in turn, minimize any risk associated with tampering their brand reputation. Carter’s 10Cs model suggests, brand reputation can be associated with control of the process, clean (has a legal license) and culture.
3. Helps in stimulating the cost factor: There are various cost factors associated with appointing suppliers. A huge sum of money is invested by organizations during the selection process. By having a systematic evaluation form, organizations can understand the overall ability of the suppliers to fulfill the organization’s expectations and requirements. Similarly keeping such records for all suppliers may save the organization money in the future, given that the organization may opt to appoint a previously evaluated supplier thus saving itself any additional evaluation costs. Associating with Carter’s 10Cs, the cost factor is in tandem with cost and monetary competence.

This tool in editable format

An example of the use of this tool

Another example of the use of this tool

The infrastructure of educational organizations vary significantly from one organization to another, as many aspects impact the needed infrastructure – such as the scientific area of the learning programmes delivered, the delivery format used, the pedagogic model followed, etc.. An organization that delivers natural science programs might need a biology laboratory, while one that delivers information technology programmes might need a computer laboratory. An organization that delivers programmes in e-learning will need a learning management system, while an organization that delivers presential programmes will need classrooms and adequate furniture. If the programmes are offered with a partial or full board approach, the educational organization might also need cafeterias and dormitories. Additionally to the infrastructure requirements derived directly from the characteristics of educational programmes offered, other infrastructure needs may arise from basic or supplementary services offered by the educational organization to their learners and other beneficiaries. This includes libraries, sports facilities,  medical facilities, resting and leisure spaces, among others.

Below we provide a list of common infrastructure elements that can be found in educational organizations:

Spaces associated with educational products an services

• Classrooms
• Laboratories
• Libraries
• Sports facilities (for programmes in this scientific/technical area of the programme)
• Vehicles (cars, boats, airplanes, etc. if pertaining to the scientific/technical area of the programme)
• Etc.

Spaces associated with supplementary services provided to learners

• WCs
• Studying spaces
• Leisure and resting spaces
• Administrative services areas
• Reprographies
• Cafeterias and kitchens
• Dormitories
• Medical facilities
• Sports facilities
• Etc.

Other Resources (Equipment, Tools, Devices, Instruments, etc.)

• Vehicles (school buses, cars, boats, airplanes, etc. – sometimes needed for pedagogical reasons according to the scientific/technical area of the education services delivered)
• Laboratory equipment, including monitoring and measuring resources
• Kitchen equipment, including monitoring and measuring resources
• Non-tangible tools and instruments:
  • Psychology and career guidance tools, such as psychometric scales and vocational surveys
  • Management tools, such as learner satisfaction and alumni tracking surveys
  • Management tools, such as learner satisfaction and alumni tracking surveys
  • Pedagogic tools, such as assessment of learning Instruments – e.g. tests, exams and assignments
• Etc.

The above list is not exhaustive and many other elements exist. It is the responsibility of each educational organization to identify the elements of infrastructure that are needed to adequately deliver the educational products and services they offer to learners and other beneficiaries, and to decide which ones they have internal capacity to provide and which ones require partnerships to be established so they can be provided by external suppliers. In any case, both EQAVET and ISO 21001 require not only that this identification is done, but that the adequate infrastructure is in place. ISO 21001 , being more prescriptive also requires that the infrastructure is adequately maintained. This includes preventive and corrective maintenance, such as adequate configuration and validation of its fitness for purpose. And it applies both to the infrastructure and the work environment related to it. Examples:

• A biology laboratory with the adequate equipment, measuring devices adequately calibrated and adequate conditions of hygiene and asepsis to allow that experiments done can provide the expected results.
• A classroom with sufficient furniture for the number of students that will attend classes in it and adequate conditions of lighting and noise reduction through double glass windows, but also through values of studying in silence and of respecting colleagues when they speak by also keeping silence.
• A cafeteria with sufficient furniture to seat the amount of students that are supposed to have a meal inn each time slot and adequate cooking equipment to cook the meals, which an ISO 22000, HACCP (Hazzard Analyses of Critical Control Points) or similar system implemented to assure food safety.
• A dormitory with sufficient furniture to seat the amount of students that are supposed to live there and security measures in place, from fire alarms to door keys and security agents, assure the safety and security of the onboarded students.
• Etc.

Given to its complexity, dedicated Guidelines to non-tangible resources are provided in this toolkit.

When educational organizations offer psychology services to learners, psychologists use scales to support their work. A few common examples are:

• Scales used to assess cognitive ability – e.g. Weschler Intelligence Scale for Children (WISC)
• Scales used to diagnose neurodivergences or functional impairments – e.g. Conners scales for ADHD, Weiss Functional Impairment Rating Scale, Autism Spectrum Rating Scales (ASRS)
• Scales used to assess vocational identity – e.g. VIS, VIM
• Etc.

Unlike physical equipment and devices, which can be calibrated to assure its fitness for purpose (as required by EQAVET implicitly and by ISO 21001 explicitly),  these non-tangible tools require a different process: The educational organizations will need to assure that only scales that have been scientifically validated are used. Particular attention should be given to the nature of each scale to determine if these are culturally sensitive. If they are, re-validation in the culture of the user group might be necessary – to be noted that a simple translation of the tool does not assures its fitness for purpose in a culture different from the one in which the tool was initially validated. Another situation that constitutes a threat to the validation of these tools is its usage partial usage – for example, by selecting items to be used. Developing a new tool with parts of different (validated) tools will not create a new validated tool – the new tool will need to go through a scientific validation process.

Educational organizations use a multitude of management tools to monitor and measure different issues. For example:

• Satisfaction – of learners and other beneficiaries, of staff, of partners
• Alumni tracking – g. to check employability rates of study programmes

These tools are usually home-developed in the form of surveys and, unlike physical equipment and devices, which can be calibrated to assure its fitness for purpose (as required by EQAVET implicitly and by ISO 21001 explicitly),  these non-tangible tools require a different process: The educational organizations will need to assure that only surveys that have been previously validated can be used. Examples of methods that can be used to do so, are:

• Validation by Expert Panels – e.g. DELPHI method
• Pre-­testing with a small group with monitored reaction

The most important activity of monitoring and measuring in education is related to the assessment of learning – in other words, determining what learners have learned. The results of these activities provide relevant information to modulate the learning services – in the case of formative assessment – and to decide on the certification of qualifications – in the case of summative assessment. Given the impact of these decisions on the conformity of the learning services, educational organizations need to trust the results provided by the assessment of learning instruments they use. However, there are many threats to its reliability and, consequently, to the validity of its results:

• Each learner may have a better or worse performance depending on the type of questions chosen for a given test/exam – e.g. due to individual talents, individual communication styles, personality, stress, etc.;
• The same student may have a better or worse performance depending on the day in which takes the test/exam – e.g. due to mental and physical heath conditions;
• Different graders may grade the same content differently – e.g. due to previous experience, halo effect, etc.
• The same grader may grade differently the same content, depending on the moment in which performs the task – e.g. due to fatigue and health conditions, halo effect, etc.

Considering the above, assessment of learning instruments should not be used without being validated. There are several statistical techniques that can be used to do so, such as the Kuder-Richardson Coefficient for muti-choice questions and the Cronbach Alpha for open questions. Both measure the internal consistency, providing results between 0­1 and the higher the internal consistency, the better the reliability of the test/exam. Additionally, non-statistical methods, such as pre-tests with monitored reaction, can be used.

In some circumstances, although technically possible, it might be unviable financially to use statistical techniques nor pre-tests to validate the assessment of learning instruments, as the re-use of the instrument will not bring return on investment. After all there is a significant difference between an exam needed to achieve a Microsoft certification, which will be applied for years all over the world; and a test used once in a secondary school located somewhere in the globe. In those case, educational organizations should at least assure a systematic approach to the process of developing assessment of learning instruments. This can be done by establishing minimum requirements regarding competence, the process and the instruments. Examples are:

• Competence requirements:
  • Educators understand the threats to the reliability of the assessment of learning instruments and, consequently, to the validity of its results; and
  • Educators know how to mitigate these threats while designing and developing formative and summative assessment of learning instruments
• Process requirements:
  • A diversify of methods are used
  • A diversify of instruments are used
  • Results are used to increment the “True Score” across time
• Requirements for instruments
  • Instruments are designed by multi-cross-discipline teams
  • Type of questions used is diversified
  • Number of questions per instrument is incremented
  • Objective grading criteria is pre-defined
  • Blind and double grading are implemented
  • Etc.

This tool presents the relevant items for the creation of new qualification profiles namely:

• The area of education and training
• The identification of the qualification code (according to the national qualification catalogue) and of the designation
• If it is a new qualification or a change
• If there are regulated jobs, connected to the qualification
• If there is the possibility of obtaining partial certifications
• If applicable, the entry into force date
• The type of structure (Short length training units – SLTU, compulsory and chosen)
• The number of hours of the technological component, divided between compulsory and chosen
• A general description
• The activities or the tasks
• The knowledge
• The Do/Doing
• The soft skills/interpersonal skills
• The code, designation and number of hours of the SLTU, specific and common to other qualifications
• Comments

This tool in editable format

An example of the use of this tool

This tool carries a mechanism that enables users to design and/or update VET programmes to better address current and future student and labor market needs, thereby providing stakeholders with the most recent information on market developments and requirements. This tool is therefore to be used as a template for the creation of the proposal of a new programme or the updating of an existing one. Most of the sections require the user to describe the new programme/update at length, including detailed information on separate sheets and attaching them to this form to compile the report.

The salient reason for this is the integration of identification of training needs within the design, development and approval process of programmes, as well as during the cyclical review process. As per college policy and procedure, the design, development and approval of new accredited programmes and the review of existing programmes are key processes of any educational institution. Interested parties/proposers need to submit a form that indicates key information that is required to be submitted in the proposal. This includes Internal and external stakeholder feedback from, for example, students, alumni, industry, employers, NGOs,  etc… This process involves collaboration between various offices/departments within the institution to ensure that the programme being developed is of the best quality and suitable for the needs of the target audience. .

The intention of this tool  is to create different platforms whereby key stakeholders in education and training work together to develop programmes that are relevant and sustainable and that recognize the input from industry.

Stakeholders including academics, students and those hailing from industry and NGOs, etc., are asked to provide feedback on the level of the programme, content, learning outcomes, delivery approach, relevance and other general comments. This data is to be consolidated input for subsequent programme development and review.

This tool in editable format

An example of the use of this tool

Another example of the use of this tool

The template for course curriculum was designed in a way to allow the educational organizations to identify critical aspects of the curricula, which not only support class planning, but also serve as a tool to communicate with learners and other beneficiaries.

The template is self-explanatory and contains fields to describe:

• Course identification
  • Course name and when applicable, the programme of study to which it belongs
  • Pre-requirements, when applicable
  • Level, by reference to the European Quality Framework (EQF) or equivalent (e.g. ISCED), when applicable
  • Form of delivery (presential, e-learning, blended)
  • Timeline (delivery dates and when applicable, reference to semesters or other timeline identification)
• Responsibilities
  • Curriculum Authors
  • Teaching team
  • Any applicable Learner Support contacts (Programme, Administrative, IT Support, etc.)
• Qualification and certification information
  • Form of certification
  • Credit System and number of credits earned, when applicable (e.g. ECVET, ECTS, etc)
  • Contact hours and, when applicable, autonomous study hours expected from learners
• Pedagogy
  • Learning objectives
  • Pedagogic methods
  • Form of learning assessment and, when applicable the weight of each learning assessment element
  • Bibliography (documents, articles, standards, books, podcasts, videos, etc.)

This tool in editable format

An example of the use of this tool

Another example of the use of this tool

The most important activity of monitoring and measuring in education is related to the assessment of learning – in other words, determining what learners have learned. The results of these activities provide relevant information to modulate the learning services – in the case of formative assessment – and to decide on the certification of qualifications – in the case of summative assessment. Given the impact of these decisions on the conformity of the learning services, educational organizations need to trust the results provided by the assessment of learning instruments they use. However, there are many threats to its reliability and, consequently, to the validity of its results:

• Each learner may have a better or worse performance depending on the type of questions chosen for a given test/exam – e.g. due to individual talents, individual communication styles, personality, stress, etc.;
• The same student may have a better or worse performance depending on the day in which takes the test/exam – e.g. due to mental and physical heath conditions;
• Different graders may grade the same content differently – e.g. due to previous experience, halo effect, etc.
• The same grader may grade differently the same content, depending on the moment in which performs the task – e.g. due to fatigue and health conditions, halo effect, etc.

Considering the above, assessment of learning instruments should not be used without being validated. There are several statistical techniques that can be used to do so, such as the Kuder-Richardson Coefficient for muti-choice questions and the Cronbach Alpha for open questions. Both measure the internal consistency, providing results between 0­1 and the higher the internal consistency, the better the reliability of the test/exam. Additionally, non-statistical methods, such as pre-tests with monitored reaction, can be used.

In some circumstances, although technically possible, it might be unviable financially to use statistical techniques nor pre-tests to validate the assessment of learning instruments, as the re-use of the instrument will not bring return on investment. After all there is a significant difference between an exam needed to achieve a Microsoft certification, which will be applied for years all over the world; and a test used once in a secondary school located somewhere in the globe. In those case, educational organizations should at least assure a systematic approach to the process of developing assessment of learning instruments. This can be done by establishing minimum requirements regarding competence, the process and the instruments. Examples are:

• Competence requirements:
  • Educators understand the threats to the reliability of the assessment of learning instruments and, consequently, to the validity of its results; and
  • Educators know how to mitigate these threats while designing and developing formative and summative assessment of learning instruments
• Process requirements:
  • A diversify of methods are used
  • A diversify of instruments are used
  • Results are used to increment the “True Score” across time
• Requirements for instruments
  • Instruments are designed by multi-cross-discipline teams
  • Type of questions used is diversified
  • Number of questions per instrument is incremented
  • Objective grading criteria is pre-defined
  • Blind and double grading are implemented
  • Etc.

The template for Grading & Feedback can be used to implement a systematic approach to grading coursework, from tests and exams to group work assignments, supporting the definition of objective criteria to grade them at the design phase and then supporting the records of the grading attributed to each learner and the feedback to be provided to them.

The template contains fields to identify the type of coursework and its elements, which criteria apply to each of them and what are the maximum grades that can be attributed to each element and its total. It can support formulas to automatically calculate the final grade. It also contains fields to record the feedback to be provided to learners so they can better understand their grade and what they can do to improve their competences.

This tool in editable format

An example of the use of this tool

The template for internal audit plan was designed in line with the recommendations in ISO 19011 Guidelines for auditing management systems, which is internationally consensually considered good practices for auditing. It therefore contains fields to characterize an audit in terms of audit context (type, criteria, objectives, scope, duration and audit team identification), as well for a detailed description of the planned activities, such as which processes are expected to be audited in each time slot, by which audit team members and towards which criteria and which auditees are expected to participate. The allocation of processes to criteria and the identification of expected interviewees has two added values:

• triggers the curiosity of the auditees to (re)read the criteria applied to the processes they’re responsible for or collaborate with, which has a pedagogic effect;

serves as quick-reference additional check list to guide the audit team during the audit and allowing for self-monitoring of their effectiveness and efficiency.

This tool in editable format

An example of the use of this tool

The template for audit checklists is supposed to be filled in two different moments:

• Before the audit: columns audit criteria should be filled before the audit and allows for a more detailed and personalized view on the criteria to be used by each auditor. If in the audit plan the criteria are usually identified by their number, here is can be identified with the content of the criteria in full (useful option for inexperienced or too experienced auditors), or with criteria transformed in questions or itineraries of expected objective evidence to be seek; or with key words or in any other way that facilitates the auditor’s job. Different auditors have different auditing styles and different needs, and audit checklists should be personalized to fit them.
• During the audit: the remaining columns should be filled while the audit is being conducted. At the field “Type of Finding” auditors are expected to categorize the finding (non-conformity, risk, opportunity for improvement, conformity, strong point) and at the filed “Supporting evidence” they should list the number of the evidence they collected (and registered at the Template for audit evidence list with a sequential number) that supports their judgement regarding that finding. In the field “Auditor’s notes” auditors can pre-draft the text of the finding as it will appear in the audit report and add any other information that might be useful for the audit team, such as those needed to triangulate information to validate the evidences. This allows for the traceability of audit findings, the justification of audit judgement and assures the transparency of the process.

This tool in editable format

An example of the use of this tool

The template for audit interviewees list is a simple template for auditors to use while conducting an audit, where they can take note of the name and job role of each auditee they interview. The information entered in this template will support as needed the discussions held during audit team meetings and will feed the audit report.

This tool in editable format

An example of the use of this tool

The template for audit evidence list is used by auditors while conducting the audit and is meant to be a factual register of all objective evidences collected during the audit. It contains fields to describe the evidence and also to categorize it as collected by interview, by observation of by analyses of documented information. It is meant to be numbered in the same sequential order in which they were collected. This numbering will be reflected in the audit checklist and will avoid auditors to have to write more than once the description of a given objective evidence in the checklist, when it serves to justify the audit judgement toward more than one criteria.

The information entered in this template will support as needed the discussions held during audit team meetings and will feed the audit report.

This tool in editable format

An example of the use of this tool

The template for audit report was designed in line with the recommendations in ISO 19011 Guidelines for auditing management systems, which is internationally consensually considered good practices for auditing. It therefore contains sections to:

• incorporate the audit plan, which will contextualize the audit and provide details for what happened at any given moment;
• identify the audit sample used in the audit;
• provide a brief summary of how the audit was conducted;
• provide a brief summary and also a detailed description of audits results – the set of findings arising from the comparison between the audit criteria and the objective evidences collected during the audit;
• state the audit conclusions – the audit judgement arising from the comparison between the audit objectives and the audit results;
• incorporate the audit evidences list.

The information contained in the audit interviewee list can be incorporated in the audit report in different ways: in the section “audit sample” as sample of staff interviewed; as entrances in the category of interviews in the audit evidence list, as an annex to the audit report (best option if signatures are collected), etc..

The audit checklist can also be added to the audit report as annex. It is very useful for the auditees as a pedagogic material matrixing audit criteria, evidences and findings. However, checklists should only be included in the annex if they are readable and do not contain notes with subjective emotional judgements (e.g. a note stating “the management review record was in a terrible mess” is not acceptable. A note stating “the management review record didn’t cover all input information required by ISO 21001 and EQAVET” is acceptable).

This tool in editable format

An example of the use of this tool

The template for internal and external communication was designed based on the ISO 21001:2018 and tabulates clause 7.4 “Communication”, which contain requirements for educational organizations to define what needs to be communicated, to whom, how, when and by whom.

The template contains fields for all this information, plus precedes the columns with these fields with one to identify and organize the information by process. This approach enables a more in-depth analyses and planning of communication needs. Considering the complexity of educational organizations, its communication needs, both internal and external, are wide and diverse, as so are their communication channels and target audiences. If the reflection needed to identify all these is done for the educational organization as a whole, it is easy to miss relevant elements here and there. However, if the reflection is done by process, a more thorough identification of all needs, channels and target audiences is facilitated.

This tool in editable format

An example of the use of this tool

The template for management review was designed in line with the requirements in ISO 21001, as those are more detailed and prescriptive but cover what is expected by EQAVET. The template provides sections to:

• identify the period covered by the review
• identify the participants in the review
• address individually each input information required by ISO 21001
• describe any decisions made as outputs of the review
• allocate the outputs to the categories stated at ISO 21001

As a whole, the template allows for preparation and reporting of management reviews, which can be performed synchronously, asynchronously or a mix of both.

This tool in editable format

An example of the use of this tool