Theoretical Base for Multidimensional Classification of Learning Outcomes in Reforming Qualifications Frameworks

This article provides analysis and a reflection on basic measurable properties of qualifications, modules, units and other groups of learning outcomes, as a theoretical base for their multidimensional classifications in emerging reforming qualifications frameworks. Learning outcomes, as the main elements of a qualification, are written and organised within units of learning outcomes and different groups, giving a transparent structure to the qualification from the user's point of view (students, employers, teachers, etc.). Units of learning outcomes, modules and qualifications have a set of basic measurable properties, for example: reference level, volume of workload, academic or professional profile and quality. Reference level denotes the depth and the complexity of the acquired learning outcomes, while volume denotes the total amount of workload in ideal conditions, and profiles the field of work and study associated with this. Quality denotes the reliability of the specified reference level, volume and profile of learning outcomes. It is particularly important to take account of these reflections when considering the impact of the implementation of reforming qualifications frameworks, quality assurance, validation of non-formal and informal learning, recognition of learning outcomes and qualifications, and other European tools in qualifications systems.


INTRODUCTION
The idea for development and implementation of national qualifications frameworks (NQFs) in most countries in the European Union and in the world originates from the aspiration to increase the competitiveness of individuals and national economy and thus to increase living standards of citizens.In May 2013, the number of such countries in the world, developing or implementing their NQFs, was 142 as indicated within the NQF Inventory [1].The envisaged roles of those NQFs are different, going from interest to reflect their existing educational and training system up to build a new system.All NQFs include classification of qualifications, which is visible as their main role already from the main definition of NQF, "national qualifications framework means an instrument for the classification of qualifications according to a set of criteria for specified levels of learning achieved, which aims to integrate and coordinate national qualifications subsystems and improve the transparency, access, progression and quality of qualifications in relation to the labour market and civil society" [2].Thus, for better development and implementation of NQFs and clear benefit to all citizens, economy and the entire society in a country, it is crucial to understand all components of qualifications, such as learning outcomes, units of learning outcomes and modules, and to analyse a minimal set of their basic measurable properties.
The second section of this article presents types of qualifications frameworks -national and transnational qualifications frameworks.The third section elaborates main components of qualifications frameworks, from learning outcomes to units, modules and qualifications.Classifications of learning outcomes, and thus classifications of qualifications, in multidimensional space are given in the forth section.

TYPES OF QUALIFICATIONS FRAMEWORKS NATIONAL QUALIFICATIONS FRAMEWORKS
Development and implementation of NQFs in many countries in Europe, fortunately comes at the time where benefits can be obtained from the great wealth of experiences in other countries.Nearly all countries in the world, large and small, rich and poor, are now developing, implementing or reforming their qualifications frameworks, with the common purpose of facilitating lifelong learning and professional mobility [1].In many countries NQFs are comprehensive, representing broad national educational standards covering all educational sectors, i.e. elementary, secondary, vocational education and training, and higher education.
When analysing the status of development, implementation and even reform of NQFs in different countries, it becomes evident to see a number of benefits for individuals and stakeholders, such as:  better communication among stakeholders, leading to better relationship between education and training system, and economic growth,  more prominent role for labour market information and greater involvement of social partners,  better understanding of qualifications, within sectors and countries,  inclusion of all individuals in education and training,  developments in all education and training sectors towards flexibility and implementation of lifelong learning policies (e.g.modularisation, credit arrangements, recognition of wider learning),  strong pressures on national qualifications systems influencing national reforms, improvement their effectiveness, use of learning outcomes and development of comprehensive quality assurance systems, which leads to greater sectoral and international trust in the reliability and relevance of qualifications,  more opportunities for development of cross-sectoral competencies embedded in the context of curricula or work practice,  increased international mobility.
With respect to their roles and purposes, there are generally two distinctively opposite types of NQFs, namely; communication or enabling, and transformational or reforming frameworks [3,4] and variety of different, country specific implementations.To be even more specific, in some literature [3] three models are indicated: communication (better describing existing qualifications); reforming (improving coherence, relevance and quality), and transformational or developing frameworks, which have no reference to the existing provision.
The communications framework model is characterized by a relatively loose approach in which the framework is more considered as a possible instrument for change rather than the agent of change itself [5].Such a framework would aim to improve information on already existent qualifications system and provide transparent data to stakeholders, policymakers and users on qualifications, progression pathways and learning and career development opportunities.From an implementation standpoint, such frameworks are implemented mostly through iterative processes, starting from the already present infrastructure, strongly influenced by stakeholders including existing educational providers.With its loose and voluntary approach, in most cases, such framework do not have a regulatory purpose and is more often found in the already established and successful educational systems where stakeholders have strong roles, and there are no significant incentives for change and reform.On the other hand, reforming and transformational frameworks aim to become instruments of change.In most cases, their implementation is centrally lead (usually by Ministries) with limited stakeholders' influence and reduced influence of educational providers who tend to prefer the status quo and tend to fight institutional and personal uncertainty brought on by reforms and changes.In their core, transformational frameworks introduce strict principles on quality assurance, qualifications coherence and relevance, and are therefore more invasive, statutory instruments whose implementation in most cases starts from the vision of the future system, rather than the present position.In order to implement such frameworks, there should be strong (i.e.economic) incentive, strategic approach and consensus of stakeholders or tradition of centralised government lead reforms with strong political determination and weaker stakeholders influence.
Reforming qualifications frameworks explicitly aim to improve or modernise the existing system by strengthening its coherence, relevance and quality.This modernisation may imply the development of new progression routes and type of programmes or change the division of roles and responsibilities of stakeholders [3].A transformational framework radically breaks away from previously existing institutional arrangements and practices.An example of a transformational framework is the first generation of South African Qualifications Framework [6].However, NQFs in Europe have been presented as communication or reforming frameworks, and not as transformational frameworks.The Croatian Qualifications Framework (CROQF) is an example of reforming qualifications framework [7].
Although most of the frameworks present or being implemented today fall somewhere in between these two distinctive poles, it could be concluded that some of the early implementers (South Africa, New Zealand and UK) as well as the relatively developed Anglophone countries would rather elect a more regulatory approach, while other early implementers (Scotland, France and Australia) as well as Nordic countries would prefer a communication (enabling) approach, one which leads to increased transparency [8].On the other hand, within developing countries which would naturally choose reforming or transformational frameworks in order to faster develop their educational systems and thus increase national competitiveness and living standard, tradition, position of stakeholders and political feasibility of such invasive reforms would make all the difference.For aforementioned reasons in such countries different scenarios and implementation strategies could be found, ranging from not so invasive politically feasible iterative processes to government imposed solutions and even policy copying with weak stakeholders' involvement and lack of more reasoned public debate, in the name of "higher" national strategic objectives (i.e.EU accession) [9,10].
While developing and implementing their qualifications framework, in many countries there are a number of challenges, such as:  often attempts to have only formal stakeholder involvement rather than real expert contribution,  the development of guidelines, tools and handbooks are often underestimated,  danger of bad assessment practices, over-defining assessment, without integration and application,  loss of trust in credibility of the education and training system, difficulty of communicating with a number of stakeholders in a wide range of institutions and agencies,  a different set of institutional and individual vested interests, and  a strong tendency in resistance of higher education communities to adopting a theory and method of based qualifications design based on learning outcomes approach.
To avoid the above challenges, it is important to create a shared understanding, building on successful traditions and practices, and maintaining trust among stakeholders and users.
Considering various concerns and limited resources available, care must be taken to prioritise, establishing a structure that can effectively take NQF forward.This brings further the idea of common initiatives among countries for development of transnational qualifications frameworks and relevant common guidelines.

TRANSNATIONAL QUALIFICATIONS FRAMEWORKS
More often than not, NQFs are being built to become a part of wider, transnational frameworks with the idea not only to set up grids of levels and descriptors but to challenge current educational, professional and sectoral interests [11].Considering transnational frameworks and their influence and referencing to NQFs, two distinct types could be recognized, namely: regional qualifications frameworks (across countries in geographical proximity) and pure transnational qualifications frameworks linking countries that are not geographically connected [12].Within the both aforementioned types of transnational frameworks two subtypes are in existence today: frameworks limited to specific sectors and those connecting NQFs.While frameworks limited to specific sectors interconnect qualifications within i.e. industry sector or educational sector such as vocational education and training, those covering NQFs tend to comprehensively interconnect all national qualifications.The rationale behind both types of transnational frameworks is mobility of citizens between countries, where quality and recognition of qualifications plays a crucial role in promoting mobility [12].Still, in order to build transnational framework, two significantly different approaches could be taken [13].One is to build a so called metaframework or `framework of frameworks` which does not have its own qualifications, but rather a set of criteria and level descriptors allowing referencing between existent qualifications within national frameworks.The European Qualifications Framework for lifelong learning (EQF) is one of the best known examples of such an approach.
On the other hand, so called unified frameworks for common qualifications tend to build transnational frameworks between different countries.This approach is however in most cases limited to only a small number of sectors (i.e.Caribbean Framework).Relationship between national and transnational meta frameworks (or national frameworks between themselves) are established through the process of referencing, which involves a comparison of NQFs in order to establish the relationship between levels, credits and qualification types and classes [12].

LEARNING OUTCOMES AND QUALIFICATIONS
There are several different ways of describing all learning outcomes that a person can achieve.In almost all countries, learning outcomes are described as the knowledge, application of knowledge (or skills), and their proven usage.The proven usage refers to the conditions in which the knowledge and skills are used, including the spatial, temporal and other conditions.Analysing how learning outcomes are described in different countries, the group that worked on the development of the European Qualifications Framework for lifelong learning suggested that all learning outcomes should be described as: knowledge, skills, and competence, to simplify their description, their complexity level, and their later recognition [2].Similar descriptions were adopted in most NQFs, which may be considered to be an optimum, understandable and measurable structure of learning outcomes.
According to the EQF [2], knowledge refers to factual and theoretical knowledge, i.e. acquired specific pieces of information and their linking together.The acquired pieces of information may include terms, their definitions and other forms of factual knowledge, that in and by themselves do not open up an unequivocal possibility of creating new information based on a limited number of the existing pieces of information.Linking together distinct pieces of information may refer to various theories, model, and other theoretical forms of knowledge that open up a possibility of unequivocal creation of new useful distinct pieces of information.Skills are categorised as cognitive (logical and creative thinking), practical (manual dexterity and the use of previously known methods, instruments, tools, and materials).Skills involve everything that facilitates an adequate application of knowledge (factual and theoretical), regardless of whether this application refers to the speed and quantity of information processing, decision-making or physical reaction, or to the behaviours and relationships with others within different social groups, or a combination of different skills.Very often, in addition to the definition of skills as in the EQF, skills include social components (establishing and developing interpersonal relationships) [14].Competence means the proven ability to use knowledge, skills and personal, social and/or methodological abilities in work or study situations and in professional and personal development.In the context of the EQF, competence is described in terms of responsibility and autonomy.
According to the EQF [2], learning outcomes means statements of what a learner knows, understands and is able to do on completion of a learning process, which are defined in terms of knowledge, skills and competence.It means that the learning outcomes denotes positively assess knowledge and skills by a competent body, in accordance with the competence (autonomy and responsibility), which a learner has achieved through learning and proves after learning process.Furthermore, in the EQF, it is stated that qualification means a formal outcome of an assessment and validation process which is obtained when a competent body determines that an individual has achieved learning outcomes to given standards [2].It means that qualification refers only to knowledge, skills and competence that have been assessed by the competent body.
Thus, we conclude that the main components of any qualification are learning outcomes of a given standard.Usually, learning outcomes in qualifications are organised within set of different levels of groups and sub-groups of learning outcomes, usually called as units of learning outcomes, modules of learning outcomes, or even partial qualifications, etc.
Unit of learning outcomes denotes the minimum complete set of related learning outcomes, and Module of learning outcomes denotes one or more units of learning outcomes with a defined and harmonised number of credits (for example, multiplied by 5: 5, 10, 15 and 20 ECTS credits, or multiplied by 6: 6, 12 and 18 ECTS credits, or any other factor) [15].

CLASSIFICATIONS OF LEARNING OUTCOMES PROPERTIES OF LEARNING OUTCOMES
In general, it is possible to introduce different properties of any set of related learning outcomes (i.e.units, modules, partial and full qualifications), such as: the year of achievement, period during which the learning outcomes were achieved, the main role, providers, individual grades, etc.However, most of reforming NQFs (such as the CROQF) introduce a minimal but complete set of measurable basic properties of any set of related learning outcomes: reference level, volume of workload, academic or professional profile, and quality 16, 17.
Among those four basic properties, one of them -the reference level -appears as the most visible in qualifications frameworks.The rest of these properties, the academic or professional profile, the volume of workload and the quality, are also equally important within NQFs, or even more so (such as the quality).For example, the volume of the workload is even an explicit part in the higher levels of the EQF (and the Qualifications Frameworks for the European Higher Education Area, QF-EHEA).For illustration, qualifications at the first cycle of the QF-EHEA (or 6th level of the EQF) should have the minimal volume of workload of 180 ECTS.
Reference level denotes the complexity of the achieved learning outcomes, independent of other basic properties (volume, profile, and quality).Volume denotes the total amount of the achieved learning outcomes and it is independent from other basic properties (reference level, profile, and quality).Profile encompasses the field of work or study, the main role, profession, and other similar characteristics of learning outcomes, and it is independent from other basic properties (reference level, volume, and quality) [16].
The quality of any set of related learning outcomes denotes reliability and credibility of the statement made by the official certificate documenting the other properties of learning outcomes.Quality is expressed in two dimensions: personal and institutional.The personal dimension of the quality describes reliability of the presence of the learning outcomes (of the given reference level, volume and profile) as something that an individual possesses, and it is usually expressed in grades or similar set of information.The institutional quality refers to the reliability of the provider.A certain value of the personal and institutional quality is implicitly involved and required for all learning outcomes (e.g., 60 % as a threshold for individual quality and satisfactory institutional system of quality assurance).The quality, unlike all the other properties, can be assigned only after the process of assessment (assessment of achieved learning outcomes for personal dimension, and assessment of internal quality assurance system for institutional dimension of the quality).
For each of these basic and complete set of properties, qualifications frameworks gives methods for their identification and for expression of their values.Reference level is determined by means of level descriptors and expressed by numbers (e.g. level 7).The value of the volume of workload is expressed in credits (e.g.ECTS), and the profile should be indicated by the specific part of the title (e.g.medicine, physics, mechanical engineering, etc.).
We can easily conclude that any set of related learning outcomes, achieved by learners at a provider with satisfactory quality assurance system, could be mathematically expressed in four-dimensional space, as follows: where LO symbolically represents any set of related learning outcomes (e.g., unit, module, qualification), R represents value of the reference level, V the volume of workload, P the profile, and Q the individual quality.Set of four vectors ( R e  , V e  , P e  and Q e  ) symbolically denotes a basis of independent unit vectors in four-dimensional space of any set of related learning outcomes.Usually, in educational systems, a set of learning outcomes are accepted only if the value of individual quality is higher or equal to 60 % (i.e.Q > 0,6), which represents the probability that the individual has achieved the set of learning outcomes (i.e.knowledge, skills and competence).
For example, the qualification of Master of Science in Biology (MSc in Biology), which a learner achieved with a higher grade (e.g. 90 %, Q = 0,9, which means a grade "A" in Bologna Process) at a university with a satisfactory institutional quality assurance system, should be written as:

CLASSIFICATIONS OF LEARNING OUTCOMES AND QUALIFICATIONS
Qualification has the same basic properties as any other set of related learning outcomes and can be classified by any combination of levels, workloads, profiles and quality, i.e. .
For example, one classification of qualifications can be done only by levels (within the dimension of R e  ), or another classification by combination of levels and profiles (within two dimensions, R e  and P e  ), etc.
Examples of such classifications we can find elsewhere.For example, one classification can be done by introducing types of qualifications (which is a combination of level and profiles, R e  and P e  ).In addition to classification by levels, classifications by types of qualifications make deeper classifications of qualifications (e.g. for level 6 qualifications, in most educational systems there are usually different profile classifications: Bachelor of Science, Bachelor of Arts, Professional Bachelor, and even "Master Craftsman", etc.).
Furthermore, independent to levels of qualifications, classes of qualifications may also be observed in existing systems (e.g.class of major qualifications, class of special purpose qualifications, class of minor qualifications, etc.), which represents classifications of qualifications as a combination of volume and profile, V e  and P e  ).

CONCLUSION
We have analysed different groups of learning outcomes and concluded that learning outcomes, as the main components of any qualification, in practice are written and organised within units and/or modules of learning outcomes, giving a transparent structure to a qualification.Units of learning outcomes, modules and qualifications have a set of characteristics, for example, year of achievement, period during which the learning outcomes were achieved, the main role, providers, individual grades, etc.But all those characteristics of any set of learning outcomes (knowledge, skills and competence) can be written as a combination of smaller set of properties -the reference level, the volume of workload, the profile and the quality.It means that any set of learning outcomes (e.g.qualification) can be mathematically written as a vector in four-dimensional space, It means that, for example two qualifications are the same only if values of all properties for both qualifiactions are the same.
Those basic measurable properties can be further used as a theoretical base for their multidimensional classifications in reforming national qualifications frameworks (e.g. in the CROQF).Clear classifications of any set of learning outcomes are important for transparency and understanding of achieved learning outcomes and qualifications in modern educational systems.Classifications of learning outcomes make further a basis for transparent definition of access and progression within the educational systems for all individuals.In addition, representation of any set of related learning outcomes in the above four-dimensional space gives also the basis for development of the quality assurance system and the system for recognition and validation of non-formal and informal learning with the principle of equal value.
For example, using the above representation of units of learning outcomes, it is possible to conclude that the equality of the value of units of the same set of learning outcomes achieved by different type of learning (formal, non-formal and informal) is only possible if the quality of those learning outcomes, achieved by individual within any type of learning, is the same.This requires further the standardisation of assessment criteria and procedures, regardless of the type of learning.Of course, within NQFs there should be no space for a number of different quality standards for the same set of learning outcomes.Before assessment takes place, regardless of the type of learning, one set of learning outcomes is fully described by the same values of three of four basic properties: the profile (indicated by a proper title), the level and the volume.In order to fulfil the principle of equal value it is theoretically clear that the remaining element of the set of basic properties (i.e. the quality in this example) should have also the same value, which is possible only if assessment criteria and procedures are standardised.