INTRODUCTION

The world is currently going through its fourth industrial revolution, which was preceded by other three critical historical processes, as follows: the first industrial revolution paved the way for mechanized production from manual production and took place between 1760 and 1830; the second industrial revolution of 1850, after electricity was invented, enabled mass production; and the third industrial revolution of the mid-20^th century, introduced electronics, information technology and telecommunications. The fourth industrial revolution, also known as “Industry 4,0”, is a combination between digital, physical and biological systems to benefit humankind’s transformation, as per Klaus Schwab, founder of the World Economic Forum. (UPB news agency, August 1^st, 2018)

Today, Industry 4,0 champions robotics, Internet applications in product management, massive use of digitalization, development of artificial intelligence, information technology (IT) and network-connected intelligent devices servicing clients around the world, according to the Global Risks Report 2016 (World Economic Forum, 2016). In this regard, the fourth revolution challenges the way in which human beings are to face the demands of the surroundings and even themselves. Nowadays, there are several examples in different fields in which technology, through artificial intelligence, has begun replacing humans in science and even liberal professions. The implementation of Virtual Learning Objects (VLOs) encourage student motivation, self-regulated learning and appropriation of concepts and knowledge related to the matter, which is why it has been proposed as a learning strategy (Parra-Esquivel, Peñas-Felizzola, & Gomez-Galindo, 2017). Andres Oppenheimer, in his book “Every Man for Himself”, states that the exceptions are bound to be: “…people with top levels of skill or study” especially those who stand out for their “…creativity, originality, and social and emotional intelligence…” (p. 21). Consequently, usability, and Virtual Learning Objects in particular, -the subject of this study- is precisely one of the areas that are destined to remain and to further develop in the world of the fourth industrial revolution.

At national level, in the department of Quindio, a cluster of companies has been working towards the same views of usability following the triple helix model of innovation: University – Industry – Government, this group has been acknowledged by the Private Council for Competitiveness as Cluster UXARTE.

Indeed, the official website of Red Cluster Colombia, highlights the cluster’s work as follows:

This work has allowed to articulate the educational offer with the software industry’s requirements in the region, strengthening the software sector; the work has been so fruitful that the vision deriving from uXarteTIC… has been recognized and supported by the national government, based on its shared and agreed outlook of different actors which, due to cohesion and trust… have succeeded in the short-term and managed to place Cluster uXarteTIC… as one of the most important in Latin America due to its infrastructure and knowledge on usability… (Red Cluster Colombia, 2019, https://redclustercolombia.com/clusters-en-colombia/iniciativa/219)

Education is precisely one of the fields in which the methodology proposed by different authors about usability is implemented with Virtual Learning Objects (VLO). In recent years, Colombia has rigorously implemented comprehensive learning processes, with its main promoter being the Ministry of National Education (MEN, for its Spanish acronym). The use of virtual objects gives students the chance to build its interpretation and organize it to his/her own cognitive structure in different interpretation stages, which indicates that new knowledge is created by processing the information incorporated from his/her own daily experience through the use of ICT (Carlos, Cardozo, & Caribe, 2018). Studies in other knowledge areas arrive to similar conclusions and concur that the use of these types of tools constitute promising educational experiences: an interactive experience, akin to reality, of dynamic and constructive learning may benefit student’s comprehension of concepts and appropriation of knowledge (Alvarez & Dal Sasso, 2011).

Currently, a considerable number of organizations and entities in the public and private sector are promoting the design and implementation of computer-based virtual objects and of Virtual Learning Objects. Among the institutions that are uptaking these educational changes along with the objectives of the Cluster UXARTE initiative, is Universidad del Quindio, an entity with its own research group and virtualization unit, with human capital trained in the area, in alliance with other entities that orient teaching processes using these objects –such as Sena, Colombia’s national training service, in Quindio, which offers diverse training areas using virtual tools–. In the academic sector, the usage of these media is considered critical since it enables the ubiquity of information, allowing access to experiences by community members anywhere and anytime, extending accessibility to learning resources stored in Learning Management Systems – LMS (Ribon, Monroy, & Marrugo, 2018).

The aforementioned in response to the transformation of teaching-learning processes undertaken by the Ministry of National Education on its educational programs as didactic strategy, disseminating virtual education and making this modality a key element for users’ socioeconomic development and competitive quality. Due to the importance of these tools for training processes, it is essential to evaluate and question the efficacy of content and interactive didactic resources used in this virtual teaching system, which displays other forms of education along with ICT.

This educational offer of the millennium must be supported by optimum virtual tools, both innovative and motivational, to facilitate the student’s meaningful learning; it is important for the user to feel that the tool is friendly, easy to understand, appealing and that it has suitable content to acquire the required competences and fulfill the learning process’ goals. This entails the use of methodologies through Information and communications technology (ICT), which become the instrument that gives users the opportunity to be engaged, capable to build knowledge through guidelines, tasks and scenarios, also known as Virtual Learning Objects (VLO).

Correspondingly, institutions that teach programs in the virtual modality must provide innovative, efficient and top-quality services that are not just based on technological tools and communications equipment, but that focus on users’ needs; as expressed by international standards for software evaluation, what drives optimum learning is:

Design quality and usefulness of content to be learned, understood, used, it must be appealing, clear and simple for the user. It is imperative for a virtual methodological tool to work properly and for the user to feel comfortable using it, meaning the interaction between user and VLO must make the acquisition of his/her objectives possible. Therefore, to evaluate its usability, the ISO/IEC 9126 standard must be considered, this is a group of standards that regulate software product quality, specifically ISO/IEC 2500 (SQuaRE), ISO 9241-10, and ISO 9241-11 (ISO/IEC 9126 Standard on software product quality).

Additionally to international standards, the study took into account the ideas of different authors on the topic, such as Alva, Gonzalez, Boklaschuk, Nielsen, Shneidrman, and others, who describe general metrics to measure usability attributes. Moreover, the “User-Centered Design methodological framework (Norman, & Draper, 1986), was applied and adapted to the typical characteristics of web applications’ development” (Tellez, 2009, https://priscillatellez.blogspot.com/2009/06/diseno-web-centrado-en-el-usuario.html).

METHODOLOGY

In order to evaluate the usability of a VLO and its characteristics, an overall analysis must be conducted by all of the units in the academic space offered under virtual modality, taking into account three proposed methodologies:

• A. Evaluation by means of ten principles and criteria proposed by author Jakob Nielsen.

• B. Evaluation by means of users, proposed by several authors such as Shneiderman.

• C. Evaluation by means of usability, based on a three-level hierarchy (metrics, criteria and attributes).

Methodologies such as these allow a holistic view of the VLO and its characteristics, this requires recognition and prior familiarization; of the site, implying freely browsing the Virtual Learning Objects (VLO);, which must clearly explain users (students) the importance of the correct use and handling of the website concerning any activity to be developed in the teaching-learning process. This will introduce an overall presentation of the course in terms of notifications, platform demos, bibliographies, glossaries, web links, content of the academic space, tutor’s details, documents, support materials, works proposed, discussion forums (social, inquiries, technical and thematic), media outlets, messaging, synchronous and asynchronous sessions, chat and grades.

Methodology A. Evaluation by means of ten principles and criteria proposed by author Jakob Nielsen,; conducted with an instrument that involves the ten principles: visibility of system status; match between system and real world; user control and freedom; consistency and standards; error prevention; recognition rather than recall; flexibility and efficiency to use; aesthetic and minimalistic design; help users recognize, diagnose and recover from errors; help and documentation (Galvez, 2016).. Recuperado de http://repositorio.unjbg.edu.pe/bitstream/handle/UNJBG/2468/924_2016_galvez_pilco_md_fain_ingenieria_en_informatica_y_sistemas.pdf?cv=1&isAllowed=y&sequence=1

These principles must be applied in general to the object of the study, enabling to realize positive or negative findings to determine usability; data obtained is processed by each evaluator (VLO user) in a matrix that is subsequently processed and analyzed in order to generate the results and conclusions to determine if the VLO is usable or not.

Methodology B. Evaluation by means of users, proposed by several authors such as Shneiderman,; conducted with a questionnaire that allows verifying if the VLO is usable or not, it takes the following into account: objective; site; generalities; content; browsing; identity and information; use of descriptive menu tabs; easy browsing; accessibility; help; adequate type of language, writing and presentation; these allow to quantify the reach of the VLO’s objectives in terms of usage and meaningful learning of the content offered by an academic space, drawing positive or negative conclusions of the site being evaluated and indicating whether it is usable or not.

Once the VLO evaluation has been processed with the two previous methodologies and based on the results obtained by the questionnaires

, the effectiveness percentage of the site’s usability is determined; this requires an analysis of favorable answers of instruments applied set above 90%. If the compliance percentage is above that amount, a thorough evaluation is suggested, for which we propose Methodology C. Evaluation by means of usability, based on a three-level hierarchy proposed by authors Claros, and I., Collazos,, C. observed in Diagram 1; , which specifically allows identifying the attributes and criteria that require inclusion of opportunities, and through these shortcomings, deliver action plans to improve the site

Diagram 1
Model of evaluation of usability based on a three-level hierarchy
Source: Claros, I., Collazos, C.

Methodology C. evaluation of usability based on a three-level hierarchy

Implementing this methodology requires designing metrics, criteria and attributes; evaluation of usability based on a three-level hierarchy based on metrics is made up of criteria, as illustrated in Diagram 2, criterios conformadosformed by a specific set of attributes (characteristics that are determined through questions of satisfaction), enabling a detailed analysis measuring the effectiveness, pertinence and hierarchy of each criteria. This valuation is obtained using the Likert scale that allocates a score between 1 to 5 depending on the importance of each attribute in the learning process, to wit: 1 represents Strongly disagree and 5 Strongly agree.

Diagram 2
Metrics and Criteria
Source: compiled by the authors, 2018 según teoría de autores

Then, the results were analyzed and each attribute is given a score,; this is weighted with the formula

, where . represents the amount of evaluators who gave the same score, . is the total of evaluators, indicating the percentage of participation; immediately after, for each attribute, the relative frequency of the scores valuated 4 and 5 given by evaluators is added; based on these results, each attribute is given a quantitative score, as seen in Chart N°1.

Chart N°1.
Quantitative score, as per the valuation scale
Source: compiled by the authorsconstruc, 2018

Weighting of each criterion comes from the average assigned to the set of attributes that evaluate a particular criterion;, taking into account Chart 1. With these results, the average of each metric is calculated for each criteria,; results above or equal to 90 indicate that the VLO complies with the usability requirements suitable for meaningful learning; those below that figure indicate that improvement actions are needed for the evaluated metric. The aforementioned proves is exemplified in Diagram 3 for just one metric.

Diagram 3
Exemplification of allocation of weighted scores to evaluate metrics, criteria and attributes
Source: compiled by the authors, 2018

The VLO’s general usability average is calculated using the measurement obtained in each metric, analyzing the metrics’ score, learning, content, communication, method, operability, attractiveness and satisfaction, as seen in Diagram 2.

If the VLO’s general usability average, by means of the aforementioned proposed methodologies, does not satisfy the requirements of the user and of the institution that offers the academic space, an implementation or enhancement of the VLO must be recommended, keeping in mind the characteristics that scored below 90%.

Finally, in case the VLO was improved or redesigned, it needs to be verified to evaluate if it complies with the metrics, attributes and criteria based on the three-level hierarchy proposed by several authors, and to confirm that it contributes to increased meaningful learning. To do so, a test must be applied in an experimental group with the improved or new VLO and in a control group with the current VLO in place at the institution, this exercise will allow identifying if relevant changes in student (user) knowledge were attained, since the student can determine whether the site expanded and satisfied his/her training and knowledge levels.

The methodology proposed to verify if significant changes have taken place with the new or enhanced VLO, is conducted by contrasting hypotheses, comparing the changes in acquired knowledge observed in the experimental and control groups.

RESULTS ANALYSIS

The purpose of the study was to evaluate the usability of the VLO used by Uuniversidad del Quindio;, the aforementioned methodologies were applied in the development of the topics of basic mathematics as the chosen academic space in the virtual modality; initially the students (evaluators/users) were offered a general outlook of the structure of this technological tool.

Later, instruments were applied to verify and evaluate usability applying Methodologies A and B, with results obtained from the questionnaires, and it was concluded that the percentage of effectiveness regarding the site’s usability was below 90%.

Unsatisfactory results regarding the usability of the VLO used by Uuniversidad del Quindio, obtained with Methodologies A and B, were subject to Methodology C, based in the three-level hierarchy; the evaluation for the learning metric with this third methodology is shown in Diagram 3, its average score was of 66.88%; likewise, other average scores were obtained for each metric, these results are illustrated in Graph 1, which shows that the general usability average as per the metrics, criteria and attributes is below 52..82%, a value that is not suitable for the tool’s usage and purpose, this result led to improvement actions and to introducing relevant changes to the current VLO.

Graph 1.
Graphic representation of each metric’s usability
Source: compiled by the authors, 2018

Changes or redesign applied to the VLO used by Universidad del Quindio were proposed based on the unfavorable result in terms of usability, this process considered implementing didactic strategies to boost the tool with the aim of fulfilling the conditions required for its use as per the characteristics described in its metrics, criteria and attributesobteniendo un OVA mejorado;. The verification of compliance regarding the improved VLO’s usability characteristics was done by making it functional and analyzing if it positively impacted students’ meaningful learning.

To evaluate student’ efficiency, appropriateness and academic progress using the improved VLO in contrast with the current VLO, a comparison was conducted through knowledge tests, these evinced the progress made by users, which had to be of similar age, academic level, technological tools and knowledge, thus delivering unbiased results. For the comparison, an experimental group was involved in activities in the academic space using the improved VLO and a control el cualgroup engaged in the same activities with the current VLO.

Before using the VLO for the development of the course, an initial test or “pretest” was run with both the experimental and control groups, with the purpose of verifying the knowledge of students in the offered academic space about topics to develop; also, it was assured that participants had the same level of knowledge before the activities began. The results are in Graph 2.

Graph 2
Results of the average grade in the pretest
Source: compiled by the authors, 2018 con base a los resultados

The average grade obtained by groups in the “pretest”, prior to the acquisition of knowledge with the course, was below 3

, which indicated students’ deficiencies in some topics of the mathematics academic space. Chart 2 summarizes averages and standard deviations from the pretest.

Chart 2
Statistical calculations of the pretest (statistical)
Source: compiled by the authors, 2018 con base a los resultados

After calculating the information of the pretest, a point estimate of the difference between the means el cualwas used, this determines if between the two groups there were significant differences in the knowledge of the offered virtual course; the point estimate of the difference between the mean grades was calculated with the formula:

= (1.5-2) = -0.5 and the standard deviation of the difference between the means as observed in

formula (1).

Subsequently, the margin of error was calculated, as follows

; considering the theory exposed by the author (Mason, 2001), who states that if

.

) > (Ce) differences in knowledge are presented by students in the two groups engaged in the experiment. For this case, the difference of the average grade is lower than the margin of error (-0.5 < 0.7118), .; it was concluded that there were no significant differences between the averages obtained by both groups, a result that allowed verifying that students had the same level of knowledge in the topics to be developed by the offered virtual course; moreover, it was confirmed that both groups began the experiment in equal conditions.

PosteriormenteWhat followed was the development of the course with the two VLO in place. The current VLO was allocated to the control group, and the improved VLO to the experimental group; also, the students (users) evaluated the stability of the tools by verifying if the quality metrics designed and presented in the article herein for the improved VLO fulfilled or not compliance above 90%, these were based on the results of basic metrics proposed by some authors and used in the initial test of the current VLO. Each group had the same number of students, for a total of 2*n students-evaluators, as seen in Graph N°3.

Graph 3
Summary of evaluation of general metrics current VLO vs. improved VLO
Source: compiled by the authors, 2018 con base a los resultados

Graph 3 shows that the evaluation of usability performed on the current VLO reveals that none of the metrics delivers a usability level above 70%, contrary to the results obtained with the improved VLO, which had an average score of usability of the metrics above 90%, after applying the weighting in Chart 1.

The analysis of the score shows that the learning metric with the current VLO had a general average score of 66.88% based on scores of each criteria (characteristics), as seen in Graph 4; a low score according to the weighting explained in Chart 1, taking into account that the tool’s key objective focuses on teaching-learning. Although some students expressed there was an ease of learning with the tools provided by the current VLO, over 50% affirm not having learned throughout the course; some of the causes mentioned included outdated content and insufficient time to develop proposed content and activities; also, they disagreed with the teaching methodology applied in the academic space.

Graapha 4.
Learning metric, current VLO vs. improved VLO
Source: compiled by the authors, 2018 con base a los resultados

The score obtained by the same metric with the improved VLO surpassed the other one by 30.37 points, although usability did not reach 100%; some students expressed disagreements with the teaching methodology used in this VLO; 12.5% of the users thought that the content was outdated.

After averaging the score of its criteria, the content metric of the current VLO had a general score of 58.14%, this result is presented in Graph 5; this is an unfavorable value, more than 50% of the students consider the VLO’s pedagogic content to be usable, due to the fact that the bibliography did not correspond to the proposed authors and the content was outdated, preventing bibliographical scans and proper searches. Additionally, students expressed that the results of the exercises lacked verifiable steps and the overall information had grammatical mistakes

Graaph 5
Content metric, current VLO vs. improved VLO
Source: compiled by the authors, 2018 con base a los resultados

A similar situation was seen with the improved VLO, the shortcomings were similar to those of the current VLO, although to a lesser extent; the general score was of 88.79% in the level of usability.

The average general score of the communication metric in the current VLO was of 57%, as seen in Graph 6, it shows the score obtained for each criteria of this metric; 50% of the users mentioned they experienced confusion based on the lack of difference between the titles in units and its content; also, messages did not present an adequate distribution of images and texts, which hindered an ease in recalling. Another factor with negative impact was that texts and graphs did not allow to understand and locate the content.

Grapha 6
Communication metric, current VLO vs. improved VLO
Source: compiled by the authors, 2018 con base a los resultados

The opposite case was seen in the communication metric’s usability score of the improved VLO, which scored 100%, meaning students considered the control and sense of communication to be adequate, as well as the form of the messages.

The general evaluation of usability of the Mmethod metric with the current VLO, after averaging the criteria score was of 43.75%, these results are included in Graph 7; lo quethis reflects that few students believe it has enough usage instructions at the moment of beginning the course; also, communication spaces were not placed in an agile way, and the loading and execution times of process information were not reported. In terms of the criterion of adaptability, the difficulty experienced by users was not having instructions to handle the platform, nor an instrument to quantify each student’s experience in the virtual classes.

Graaph 7
Method metric, current VLO vs. improved VLO
Source: compiled by the authors, 2018 con base a los resultados

In the analysis of the improved VLO, the method metric showed a high average general score of 91.25%. However, factors preventing the score to be of 100%, were the lack of information regarding loading time, measurement of experience levels with virtual classes by students and the execution of processes.

For the current VLO, Graph 8 illustrates that the operability metric reached an average general score of 55.72% of usability based on its criteria; 50% of the students consider that it lacks undo or redo tools, and that it does not indicate routes to follow when correcting errors. On the other hand, the level of unconformity in terms of customization of the VLO is high,; students reported deficiencies in the links that minimize browsing, a sequence of steps for the learning process is also missing.

Graaph 8
Operability metric, current VLO vs. improved VLO
Source: compiled by the authors, 2018 con base a los resultados

The improved VLO obtained a significant result regarding its operability metric with an average general score of 82.6%. However, as with the current VLO, shortcomings were observed in tolerance of errors and customization; although significant warning mistakes were displayed to identify the problem, they lacked the route to correct it. Additionally, 12.5% of the students manifested that this learning tool failed to have links to minimize browsing time and over 50% mentioned it did not offer the chance to establish a sequence of steps for suitable learning.

The attractiveness criteria of the current VLO had the lowest score in terms of usability, as seen in Graph 9;, the reason for this is that less than 50% of the students considered the interface and its images to be appealing and attractive, this included text and graphics as well. They added that the font and size are not right, colors and backgrounds are neither visually noteworthy nor inspiring to browse.

Graaph 9.
Attractiveness metric, current VLO vs. improved VLO
Source: compiled by the authors, 2018 con base a los resultados

The attractiveness metric in the improved VLO had an average general score of 72.5%; this result was driven by the fact that 12.5% of the students considered the VLO’s images not to be pleasant, and only 50% said that the VLO allowed customization of the interface.

Finally, the general average of usability of the satisfaction metric for the current VLO was of 47.78%, as seen in Graph 10, this result validates the impact of every aspect of the analyzed metrics. This metric’s consists of three criteria: reliability, physical satisfaction and acceptability, all of which had low scores; the one with the lowest score was physical satisfaction, with 20%; acceptability was affected due to the fact that less than 70% of the participants considered that the design of this methodological tool increases learning, and only 62.5% expressed that the information provided is reliable and may be verified afterwards.

Grapha 10
Satisfaction metric, current VLO vs. improved VLO
Source: compiled by the authors, 2018 con base a los resultados

Concerning the improved VLO, the satisfaction metric had an average general score of 85% of usability based on the evaluation of its criteria;, acceptability was the only criterion that achieved 100% of students’ approval. Also, 87.5% of them added that the information provided could be verified afterwards. Physical satisfaction got a score of nearly 70% because some evaluators mentioned that the VLO’s experience failed to stimulate attention and 12.5% said that working with these methodological tools is not motivating.

The final result of the average general evaluation of usability of the current and improved VLO after an analysis of each of its metrics is illustrated in Graph 11, the score that leads to conclude that the redesigned VLO exceeds the average score by 35.38% in terms of the current’s VLO usability.

Grapha 11.
Average usability score, metric, current VLO vs. improved VLO
Source: compiled by the authors, 2018 con base a los resultados

Finally, to determine if the usability of the VLO has been accurate or not in terms of effective learning, a (knowledge) “posttest” was applied to the students in both groups who took part of the experiment and interacted with the VLO. The purpose of this final test was to identify the pedagogical effectiveness of both VLO from the point of view of understanding the content of the course offered. A roundup of the results is found in Graph 12.

Graph 12.
Results of the average grade in the posttest
Source: compiled by the authors, 2018

The average results concerning the learning levels shown in the grades of the final test, explained in Graph 13, verify that the students who used the improved VLO had a higher learning percentage than those using the current VLO. Although every student managed to increase learning, it is evident that the level of usability of the VLO in the experimental group contributed to the meaningful learning of the curricular content of the course offered.

Graph 13.
Learning level metric, current VLO vs. improved VLO
Source: compiled by the authors, 2018 con base a los resultados

The statistical data of the grade obtained in the posttest by the two groups is summarized in Chart 3; lastly, this information was used to contrast the hypothesis to verify if significant changes in learning took place with the improved VLO against the current VLO.

Chart 3.
Statistical calculations of the posttest score
Source: compiled by the authors, 2018 con base a los resultados

The contrast of the hypothesis was done by applying the theory of authors Richard I. Levin, and David S. Rubin (2004, 320), the technique of contrast of hypothesis for small samples using the student's t-distribution, concepts that were applied by this study since the size of the sample is below 30. The statistic value calculated to verify the hypothesis and that corresponds to a student's t-distribution is shown in

formula (2).

This research’s general hypothesis is made up of a null and an alternative hypothesis, as follows:

Null hypothesis (H... Estudents’ average meaningful learning for certain course with the improved VLO (._experimental) does not differ from the average meaningful learning developed in the academic space with the current VLO (._control); symbolically, the null hypothesis is expressed as follows:

Alternative hypothesis (H... E students’ average meaningful learning for certain course with the improved VLO (._experimental) differs from the average meaningful learning developed in the academic space with the current VLO (._control); symbolically, the alternative hypothesis is expressed as follows:

The graphic representation of the hypotheses is:

The decision rule for the general hypothesis proposed before was that the null hypothesis was rejected if the amount of calculated . was below -2.0930 or above 2.0930, as ._calculated = 4..188 was above 2..0930, then the null hypothesis was rejected and thus the alternative hypothesis was accepted; it can be stated with a confidence level of 95% that meaningful learning achieved by students enrolled in the course offered with the improved VLO (._experimental) was different and better than that of students that used the current VLO (._control).

CONCLUSIONS AND RECOMMENDATIONS

• ,The function of usability is to achieve users’ effective interaction, it must be easy to use, flexible, quick, simple without being boring, induce research, and provide updated information; the simpler and more complete the tool is, the higher the likelihood to attain the objectives of the student’s academic training.

  Usability of virtual tools defined based on quality components to reinforce meaningful learning, such as: efficiency, portability, satisfaction, productivity, simple handling, efficacy, maintenance, pertinence of didactic strategies to drive knowledge are factors that need to be evaluated and frequently updated by institutions offering virtual learning platforms for this didactic strategy to deliver its purpose, monitoring users’ increased learning.

  · The usability of a website is determined by a set of satisfaction parameters that go beyond verification of access; every component must be inquired into to sustain clear criteria that determines the pertinence, as per the objective for which the VLO was designed. Therefore, different teaching resources in education have to be subject to evaluation following methodologies such as these, and new evaluation criteria must be frequently looked into.

  A VLO’s evaluation of usability raises awareness on the quality that this tool needs to fulfill the teaching-learning objectives, it must follow the ISO-9126 standard, which refers to the software product quality and ISO-14598 standard, which refers to the software product quality evaluation.

  In terms of a VLO’s usability and its learning ease, it is necessary to keep in mind that the VLO must be simple and practical to operate both for experienced users and for those with no experience. Special attention needs to be placed on user satisfaction and content organization.

  · Tools, activities, links, forums, videos, content and material of a VLO have to be visible, friendly and understandable, and must contribute to the teaching-learning process.

  · It is deemed convenient for every VLO to include a guide, help functions or a handbook to work through the course, to allow interaction with the available tools and to avoid obstacles in the process favoring the normal development of the activities.

  · It is important to frequently update the VLO’s design, especially the visuals and pedagogical content. The implementation of competitive strategies in the service offered call for innovation, with the goal of positioning the course and staying in the market, guaranteeing quality and trust in each process.

  · Evaluating a VLO’s current usability status allows it to fulfill users’ requirements, which are mostly individuals with heterogeneous learning, interested in the objectives of the institution offering top quality training through virtual channels, this guarantees meaningful learning and increases professional competitiveness. Consequently, this document proposes methodologies to evaluate VLO that were designed with the aim of having a more complete instrument to measure and make reliable decisions regarding this teaching-learning tool and of implementing it with decreased risks in different academic spaces,; thus providing education that fits the professional profile and the environment’s demands. garantizando, el

  · The development of usability strategies and tools could be further developed to the extent they get framed within a regional competitiveness framework through the cluster initiatives –Uxarte– and applying the triple helix model of innovation: University – Industry – Government.

REFERENCES

Acevedo, M. (2018). ¿Qué es la cuarta revolución industrial? Agencia de noticias UPB. Medellín: Universidad Pontificia Bolivariana. Recuperado de: https://www.upb.edu.co/es/noticias/que-es-la-cuarta-revolucion

Alcalá Calder, J. R. (s.f.). Latinoamericana de Tecnología Educativa. El papel de las Tic TIC en la animación a la lectura. Latinoamericana de Tecnología Educativa, 3(1), 395 -415.

Arnao, M., & Gamonal, C. (2016). Lectura y escritura con recursos TIC en Educación Superior. Evaluación de la competencia digital. Innoeduca. International Journal of Technology and Educational Innovation, 2(1), 64-73. doi:http://dx.doi.org/10,20548/innoeduca.2016.v2i1,1046.

Alva, M.; Martiínez, A., Cueva, J., Sagástegui, H., & Lóopez, B. (2003). Comparison of methods and existing tools for measurement of usability in the Web. In: Lovelle J.M.C., Rodríguez B.M.G., Gayo J.E.L., del Puerto Paule Ruiz M., Aguilar L.J. (Eds) Web Engineering. ICWE 2003. Lecture Notes in Computer Science, 2722, 386-389.

Alva, M., Martínez, A., Cueva, J., Sagástegui, H. (2003). Usabilidad: medición a través de Métodos y Herramientas. Readings in Interacción. España. Recuperado de: https://www.academia.edu/486986/DEFINIENDO_UNA_ESTRUCTURA_DE_EVALUACI%C3%93N_PARA_MEDIR_LA_USABILIDAD_DE_SITIOS_WEB_EDUCATIVOS

Alvarez, A. G., & Dal Sasso, G. T. M. (2011). Virtual learning object for the simulated evaluation of acute pain in nursing students. Revista Latinoamericana de Enfermagem, 19(2), 229–237. https://doi.org/10,1590/S0104-11692011000200002

Ben, Shneiderman, B., & Plaisant, C. (20131987). Designing the User Interface: Strategies for Effective Human-Computer Interaction. E.E.U.U.:Addison-Wesley. “Los 8 principios de usabilidad del Software”.

Camps, A. (2003). Una secuencia didáctica para aprender a escribir. Barcelona: Grao. Recuperado el 25 de septiembre de 2016

Carlos, J., Cardozo, G., & Caribe, S. U. E. (2018). ¿Uso de Vvideojuegos en un Ccontexto Mmultigrado? ¿Desarrollarlas Mmediante el Uuso de Vvideojuegos en un Ccontexto Mmultigrado? Panorama, 12(23), 7–17. https://doi.org/doi:http://dx.doi.org/10,15765/pnrm.v12i23,1191

Calume, R. C. G., & MejiaMejía, J. F. M. (2018). Interactivity in digital contents: Virtual learning objects and MOOCs . MOOCs. In: R. R.H., G. C., J. O., R. M., R.-G. G., L. T., …., … M. J.F. (Eds.), 2nd International Conference MOOC-Maker, MOOC-Maker 2018, (Vol. 2224, pp. 106–111). Corporación Uniremington, Calle 51 No. 51-27, Medellin, Colombia: CEUR-WS. Retrieved fromRecuperado de: https://www.scopus.com/inward/record.uri?eid=2-s2,0-85055528898&partnerID=40&md5=c9d0931c01dd48a10856a215cc02a682

Ceballos, O., & Mejía, L.I, M. L. (2016). Metodología de Mmedición y Eevaluación de la Uusabilidad de un OVA. Armeinia: Ed. Granada., Quindío, Colombia. Recuperado el 23 de 03 de 2017

Gálvez, M. (2016). Técnica basada en métodos heurísticos para la evaluación del grado de usabilidad del sitio web “El postulante”. Tesis de grado. Universidad Nacional Jorge Basadre Grohmann, Tacna, Perú. Recuperado de: http://repositorio.unjbg.edu.pe/bitstream/handle/UNJBG/2468/924_2016_galvez_pilco_md_fain_ingenieria_en_informatica_y_sistemas.pdf?cv=1&isAllowed=y&sequence=1

González, R.E.R., Arteaga, J.M., Rodríguez, F.J.A. (2007). Evaluación de objetos de aprendizajes a través del aseguramiento de competencias educativas. Virtual Educa Brasil

González, M. (2003). Evaluación de software educativo: orientaciones para su uso pPedagógico. Proyecto Conexiones Colombia.

Levine, D., Krehbiel, T., & avid S.RubinBerenson, M. (20014). Estadística para administradoresadministración. Séptima edición. México, D.F.: Pearson Educación - México.

Marqués Graells, P. (28 de 12 de 2012). Impacto de las Tic TIC en la educación: fFunciones y limitaciones. tres 3 ciencias, 2 -15. Recuperado el 28 de agosto de 2016de: https://www.3ciencias.com/wp-content/uploads/2013/01/impacto-de-las-tic.pdf

Mason, R. D., Lind, D., & Marshal, W. (20031). "Estadística para aAdministradoresción y eEconomistasía". Ambos de la Universidad de Toledo. Ohio U: SA, México, D.F.: Alfaomega Grupo Editor.

Nacional, M. d. (2015). Derechos Básicos de Aprendizaje. Recuperado el 02 de 04 de 2017

Nielsen, J. (1993). Usability Engineering. Boston: Academic Press Professional, Boston, MA.

Parra-Esquivel, E. I., Peñas-Felizzola, O. L., & Gómez-Galindo, A. M. (2017). Virtual objects for self-regulated learning of occupational therapy students. Revista de Salud Púublica, 19(6), 760–765. https://doi.org/10,15446/rsap.V19n6,62966

Pujol, J. f. (1978). Los métodos de enseñanza uUniversitaria. Navarra: Universidad de Navarra. S.A. Recuperado el 1 de octubre de 2016

República, C. d. (2009). Ley 1341. Principios y Reglamentos de la Sociedad de la Información. Bogotá: Gobierno Nacional de Colombia.

Ribón, J. R., Monroy, M. E., & Marrugo, P. P. (2018). Integration of virtual learning objects in immersive E-learning communities. In: 16th LACCEI International Multi-Conference for Engineering, Education Caribbean Conference for Engineering and Technology, (Vol. 2018-July). Universidad de Cartagena, Colombia: Latin American and Caribbean Consortium of Engineering Institutions. https://doi.org/10,18687/LACCEI2018,1.1,142

Ruiz, R., Muñoz, J., & Álvarez, F. (2007). Evaluación de objetos de aprendizaje a través del aseguramiento de competencias educativas. Virtual Educa Brasil. Recuperado de: http://e-spacio.uned.es/fez/eserv/bibliuned:19233/n03ruizgonz07.pdf

Téllez, P. (2009). Diseño web centrado en el usuario: usabilidad y arquitectura de la información. Recuperado de: https://priscillatellez.blogspot.com/2009/06/diseno-web-centrado-en-el-usuario.html

Thorne, C. M. (2013). Efecto de una plataforma virtual en comprensión de lectura. Revista de Psicología, 31, 3- 35. Recuperado el 23 de agosto de 2016

Vásquez, A. (1 de junio de 2016). Lectura y escritura con recursos TIC en Educación Superior. Evaluación de la competencia digital. (InnoEduca, Ed).