Monográfico
Educating Urban Designers using Augmented Reality and Mobile Learning Technologies
Formación de Urbanistas usando Realidad Aumentada y Tecnologías de Aprendizaje Móvil
Educating Urban Designers using Augmented Reality and Mobile Learning Technologies
RIED. Revista Iberoamericana de Educación a Distancia, vol. 20, núm. 2, 2017
Asociación Iberoamericana de Educación Superior a Distancia
Recepción: 09 Diciembre 2016
Aprobación: 21 Enero 2017
How to cite this article: Redondo, E., Fonseca, D., Sánchez, A., y Navarro, I. (2017). Educating Urban Designers using Augmented Reality and Mobile Learning Technologies. RIED. Revista Iberoamericana de Educación a Distancia, 20(2),
pp. 141-165. doi: https://doi.org/http://dx.doi.org/10.5944/ried.20.2.17675
Abstract: This paper describes an educational experience using augmented reality (AR) on mobile devices as a tool for learning urban design concepts and specifically for architecture degree students. In a real project-based exercise, the participants had to design a sculpture to be placed in a public space, checking the suitability of the object as for example the form, scale, location, materials, etc., and taking into account the surroundings. The project is controlled on-site using AR on mobile platforms, encouraging collaborative learning by sharing the 3D models of their proposals, and acting both as producers and consumers of AR content in the process. To assess both the usability of technology, as well as the learning improvement, the class was divided into two groups with equivalent pre-course grades: a control group, who followed the conventional course in the laboratory, and a test group, who used AR technology. At the end of the course, the AR-using group showed a significant increase in academic performance, higher motivation and satisfaction compared to the control group.
Keywords: visual learning, architecture, urban planning, information technology, computer- assisted design, urban development.
Resumen: El presente artículo describe una experiencia educativa usando Realidad Aumentada (RA) en dispositivos móviles para el aprendizaje de conceptos de diseño urbano en estudiantes del Grado de Arquitectura. A tal efecto, se ha diseñado un ejercicio práctico sobre la casuística de un proyecto real, donde los estudiantes deben diseñar una escultura para una plaza pública en función de los parámetros del entorno, debiendo controlar sus parámetros, la forma, escala, localización, materiales, etc. El ejercicio se visualiza y controla mediante plataformas móviles de RA, lo que permite enfatizar un aprendizaje colaborativo mediante el estudio de las propuestas del resto de estudiantes in situ. Para la evaluación del proceso, tanto a nivel de usabilidad como de mejora del proceso educativo, hemos dividido a los estudiantes en dos grupos: uno primero de control, que cursó un sistema tradicional de la asignatura en base a ejercicios de laboratorio, y un grupo experimental, que utilizó el sistema descrito en la ubicación real del proyecto. Al final del curso, y en base a los resultados obtenidos y discutidos en el presente artículo, el grupo experimental obtuvo mejores notas finales, al mismo tiempo que se constata un incremento en el grado de satisfacción y motivación de los estudiantes que han utilizado la propuesta experimental. Este aspecto nos permite reafirmar la utilidad del método en la mejora educativa en el ámbito donde se circunscribe la experiencia.
Palabras clave: aprendizaje visual, arquitectura, planificación urbana, tecnología de la información, diseño asistido por ordenador, desarrollo urbano.
Urban space understanding and visual education are key aspects for architects training. They are used to make an exercise of synthesis and complicity with the environment prior to any architectural intervention. This article presents a case study based on the virtual modeling of urban sculptures for their visualization in a real environment. To carry out this educational experience we used mobile devices and Augmented Reality (AR) technology as a working tool for both Mobile Learning (ML) and the most innovative approaches to Web 3.0. It focused on architectural interventions in urban spaces and aimed to evaluate the academic performance improvement of students using these technologies. For this purpose a control group and two experimental groups were used, one per operating system (IOS and Android). Results suggest that experimental groups obtained significant improvements in their ratings compared to the control group. In this case, human-computer interaction seems to have contributed to a better understanding of basic concepts such as the scale and position of the sculptural elements in the urban space.
Urban morphology is constantly evolving: buildings and urban spaces are built, demolished, and remodeled, altering the landscape of our cities through the collective action of their inhabitants. Urban Plans describe what we want our cities to be, undergoing revisions as we change the vision of the future of our cities (Valls, Garcia-Almirall, Redondo, & Fonseca, 2014). The analysis of the complex interaction throughout time between what the city is and what it wants to become allows urban planners and designers to understand both the past and present states of the city to plan for a better future.
Nowadays, urban Planning and Urban Design are complex processes that involve actors from different disciplines collaborating and managing heterogeneous data from different sources. The visual representation of data and city elements allow the exploration of complex datasets, and can improve the collaboration of the teams involved through better communication (Zlatanova, Itard, Kibria, & Van Dorst, 2010). However, the representation of urban phenomena in three dimensions is challenging. While advances have been made in the realistic representation of cities (Zeng, Wonka, & Van Gool, 2007) or data captured from reality (Gruen, Behnisch, & Kohler, 2009), the use of the third dimension for aesthetic purposes to convey information through the abstract representation of volumes ( Dönner, Baumann, Buchholz, 2006), or the generalization or simplification of the 3D geometry (Semmo, Trapp, Kyprianidis, & Döllner, 2012), remain less explored.
Information and Communication Technologies (ICTs) such as AR can play an important role in improving the urban planning and design workflow, giving professionals new tools to evaluate the outcomes of their decisions before they are materialized, a crucial advantage since urban transformation processes can span decades and consume enormous (public and/or private) economic resources. In this paper, the authors describe an educational experiment training the professionals that will shape the cities of the future, using AR in an urban planning and design course, and assess the students’ academic performance improvement and their satisfaction using this technology. This educational research project arises within the research group Architecture, Representation & Modelling (AR&M), an intercollegiate group of researchers in the architectural field coordinated by the Technical University of Catalonia – BarcelonaTech, (UPC). The team is composed of lecturers and researchers from the Architectural Representation and Visual Analysis I & II Departments (UPC) and La Salle Barcelona campus (Ramon Llull University).
The paper continues with a brief explanation of the current theoretical framework in the background section concerning the urban development, its education and the use of ICTs in architecture and urban formation. The following sections describe the design of the study, the method and the main results obtained. Finally, we can find a conclusion section and the main future lines of research suggested.
BACKGROUND
Barcelona public space design and the role of sculptural elements
In the preparation for the 1992 Olympic Games in Barcelona, many construction works were executed. The large majority were large urban infrastructures and sports facilities, resulting in the creation and/or redefinition of new urban spaces each with their own character, where the placement of modern sculptures acquired great prominence. This type of intervention (Hortet 1987), defined as “metástasis processes” in urban proposals, has been one of the methodological contributions of Barcelona´s urban design to the world.
Following this strategy, many public works incorporated modern sculptures to provide monumental qualities to urban spaces. The monuments therefore acquired a double significance –commemorative and rememorative– beyond the purelyaesthetic presence of the sculpture (Monclús, 2004).
More than fifty sculptures were placed in Barcelona (Sadurni 2002), both from Spanish and internationally renowned authors. The objective was to beautify the urban landscape and bring art closer to people, following the spirit and tradition started in the universal exhibitions to symbolize modernity (Cartes, 1997). Catalan artists (Miró, Tàpies and Brossa) along with prominent Spanish (Chillida, López and Hernández) and international authors (Oldenburg, Roy Lichtenstein, and Carr) were involved in this process.
Teaching the skill of designing outdoor spaces in urban environments is not a common practice in architecture schools outside Spain; in the rest of Europe, this discipline together with Urbanism is part of a different degree, usually designated as Urban Planning and Urban Design.
Moreover, in other Spanish architecture schools, this subject matter is not usually as central as in the case of the School of Barcelona, which has about 50 ECTS (European Credit Transfer System) credits devoted to the matter in its architecture degree, whereas other faculties spend around 20 ECTS credits (Wallet, 2002).
As Payne (2012) proposes, visual perception is a fundamental human activity and a vital component in arts education. It is very important to explore the pedagogical implications for teaching visual perception, including the appropriate selection of media to construct and communicate the proposal. When incorporating a new artifact in a public space (sculpture and urban furniture) an exercise of synthesis and complicity is essential. This forces the architect to understand the sculptors’ (or designer´s, in the case of urban furniture) perceptual scale of the urban space, as well as the possible uses (Trias, 1976) and aesthetics of the proposal in its urban setting, being necessary to comprehend the relationship between all the elements in the visual project (Adkins, 2014).
In higher education in the fields of architecture, urbanism or building engineering, space visualization and conceptualization are essential aspects that students must master before initiating their professional career (Leopold, Górska, & Sorby, 2001). Tools that use computer-assisted design (CAD) technologies, geographic information systems (GIS) and, more recently, building information modelling (BIM), help to create virtual models that are nearly identical to actual structures and have great capacities for architecture management and teaching discussion (Fonseca, Martí, Redondo, Navarro, & Sánchez, 2014).
Education anytime and anywhere: Mobile Learning
Nowadays, the use of ICTs, and especially mobile and wearable devices (Smartphones, tablets, smart watches, smart glasses, etc.) comprise a set of tools and applications that allow the incorporation and development of new strategies in all fields of modern life: education, social relations, work, leisure, etc. In the case of education, most of the proposals developed assume that they ease the learning process and provide better learning performance based on the academic results metric. All these approaches, which intend to improve the student learning process, should be capable of providing support and address the difficulties that arise when the student interacts with new devices and applications.
We consider Mobile Learning (ML) a subfield of distance education, which includes the e-learning strategies based on the use of computers and networks. ML can go a step further by enabling teaching via wireless networks and mobile devices, allowing learning to take place anywhere and ensuring that teacher-student interaction always exists (Tsvetozar, 2004) and enabling situational teaching, where the device detects its context and provides the relevant information.
The first works on ML from a scientific point of view were approached by COMTEXT (Kristoffersen 2000), a virtual learning environment using mobile devices. Other experiences extended the same idea to a virtual university based on the Internet and mobile devices, developing a platform called Welcome ML (Wireless E-Learning) considered central by several authors (Navarro et al., 2012). This concept is developed further when ML incorporates Ubiquitous Learning (UL), where the data is stored in the cloud and can be consumed anywhere through all kinds of educational applications and/or social networks.
In combination with a collaborative model (Li, Li, & Wang, 2006), participation and sharing within the network can generate new knowledge (Naismith, 2004). The use of touch-screens, smartphones and medium-sized tablets equipped with high- speed wireless connections, GPS and long-lasting batteries, has allowed to develop new pedagogical strategies, despite the size limitations of the screens.
The applications of Augmented Reality
The constant demand for smart devices for commercial and recreational uses, as well as advances in technology, has allowed the cost of mobile devices to drop, allowing greater access to them by end users, and to increase their capabilities. Mobile devices are currently capable to provide quality real-time 3D content.
In AR applications, the user sees the real world with overlaid virtual objects; it is not a system to replace the real world as in Virtual Reality (VR), but it enriches the user perception with information. The result can be an image of a virtual 3D model shown to the user on the screen of the device used (computer, projector, whiteboard, goggles, tablet, or smartphone) inserted into the environment or live-video feed.
AR and VR share some common features such as immersion, navigation and interaction (Denleavy, Dede, & Mitchell, 2008). However, AR has two main advantages over VR (Sánchez, Redondo, & Fonseca, 2012):
Emerging technologies in education
These technologies have been recently introduced in various commercial fields with many expectations for the near future. In education there have been experiences teaching mathematics and geometry (Kaufmann, 2002) and more recently focusing on education and visualization of 3D models (Martín-Gutierrez, 2010).
The study of the relationship between student interaction, degree of satisfaction, and usability when teaching using new educational methods is extensive, with recent contributions that have helped to design e-learning and mobile experiences and dislocate teaching using ICTs (Sun & Hsu, 2013; Giesbers, Rienties, Tempelarr, & Gijsealers, 2013).
These technologies, allow the creation and use of new Virtual Learning Environments (VLE), and Virtual World Environments (VWE), new platforms to improve the education in a collaborative and interactive way (Park, 2011).
In Architecture and related areas, several contributions can be found: using mobile devices to check the final appearance of a design, urban planning and design (Matsumoto 2008), access to advanced contents in Cultural Heritage monuments (Haugstvedt & Krogstie, 2012), increase the involvement of citizens in the assessment of urban proposals ( Fonseca, Valls, Redondo, & Villagrasa, 2016), and human- computer interaction research (Erskine, Gregg, Karimi, & Scott, 2015).
Finally, digital technology can allow professionals and students to generate and modify virtual proposals quickly and easily, and in conclusion to explore the relationship between art and place (Sinker, 2013), generating new visualization methods that surpass photomontage techniques in architectural representation (Wang, 2011). These procedures are helping to address traditional problems in urban design, such as the relative scale of the different compositional elements, their number, arrangement and the control of the final design visual appearance.
Experimental design
It is possible to categorize the perception of urban elements using references of public space designers (Burton-Chellew, Mouden, & West, 2016). Monuments are one of these elements, which are perceived at urban or local scales depending on their size and location, and whose materials and details can make public spaces welcoming and adapted to the human scale.
The main objective of this educational experiment was the study of human- computer interaction when defining the scale and position of sculptural elements and street furniture in the design of outdoor spaces. This approach is novel in the fields of architecture and urban planning where the classical representation of the designs via printed plans and physical models still prevails.
The scientific treatment of the experiment was based on the study of AR technology on mobile devices and the assessment of the improvement of student academic performance. This approach complements previous works where VR has been used as an empirical research tool in educational environments (Kuliga, Thrash, Dalton, & Hölscher, 2015).
The primary working hypothesis was that using AR applications in mobile devices improved the students’ final grades. It was expected that they would prefer innovative training using technological systems (AR and ML) instead of conventional training, and that they would be more motivated, even if they had to devote more time to working on the proposed practical exercise. The secondary working hypothesis was that using these advanced visualization methods for architectural and urban proposals, the students would improve their spatial skills, which are one of the most important competences to develop throughout their training.
To test these hypotheses, the degree of usability of the application by the students was assessed. These common pitfalls should be easier to avoid when students have been physically on site, and have taken into account the perceptual impact of their proposals with the help of these technologies, using self-evaluation.
Technological framework
The experiment relied on new visualization strategies using AR in outer spaces using multiple mobile devices (smartphones and tablets) using commercial (ARmedia©) on iOS devices and free customized applications (U-AR on Android devices).
U-AR was specifically developed for this project because it was considered necessary to enable the creation and management of virtual personalized content through AR, offering advantages in the learning process, compared to other existing commercial applications. This tool is based on optical image recognition of elements in the environment that act as markers, allowing to choose up to five different ones. Each of those images can be associated with various 3D models without changing the marker, allowing manipulation, positioning and scaling of several of them. This way the user can compare alternative proposals with a single marker, such as the stages of development of a constructive process, or any other information which can be separated in layers. Another advantage is that certain model objects can be designated as occluders (Redondo, Fonseca, Sánchez & Navarro, 2012), allowing a much better model integration in the environment.
CASE STUDY
Course and participants profile
The experiment was developed with the participation of the students in the Computer Applications (3 ECTS credits), and Architectural Representation III (5 ECTS credits) courses. The primary objective was to provide the training and technical resources to allow students to design, analyze, develop and present architectural designs under the general theme of the course on Housing and Urban environment. This was accomplished using two different strategies: (a) using digital image processing techniques, and (b) creating 3D virtual environments in different formats, including AR.
It required optimizing learning time with emphasis on the generation of urban models for the proposed intervention as well as virtual scenarios, which were later evaluated from an analytical and visual communication points of view.
To validate the work hypothesis, the case study included a group in a semester course with an intensity of two-hour sessions per week. The subject of this course revolves around learning to use advanced software applications, and as a course in the junior year, is the culmination effort of all the knowledge that the students have acquired during their previous formative years.
The 17 participants in the test group and the 8 belonging to the control group were asked to answer a questionnaire to find out their prior knowledge on AR technology and technological profile ( Figure 1).

The choice of urban setting
The central theme of the course was the study of interventions in the urban landscape of Barcelona, and the case study was on Flassaders Square (Figure 2), which had recently been completed with the construction of the extension of the Picasso Museum in 2009.

In the prior courses to the case study described in this paper a variant of the exercise had been used without using AR technology, where the students had placed several works by the painter Picasso in the same location. The main problems detected that the new strategy tried to address were the following (Figure 3):

Once the location was defined, some preliminary feasibility studies (Figure 4) were conducted to assess the suitability of the technology.

Experiment phases
The goal of the course was to design a sculpture to be placed in a public space. All students made an architectural proposal (creating content using rendering and modeling packages) as in previous editions of the course, and were asked to revise their proposal after visiting the site. In this visit only the subjects of the experiment used AR technology.
Both proposals (before and after the visit) were evaluated regarding the quality of the models produced (textures and lighting, but not polygon count or texturing), site integration (scale, location, materials, form) as well as the subjective opinion about the quality of the proposal.
The course was broken down into three main phases (Figure 5):

Initial content creation and capture
In this phase the students used the digital architectural technology skills acquired throughout their training, and parametric design using SketchUp© dynamic components and rendering using Artlantis© were introduced. The aspects followed by the students are listed below and are further described afterwards:
As a first step, the students took a series of pictures of each of the buildings in the proposed environment, which were subsequently rectified as orthophotos through various specialized applications (PTlens and Photomatch), obtaining the approximate dimensions of each of the elements defining the facades.
With this information and cartographic data from the Barcelona City Council, Non-Photorealistic Rendering and CAD were used to visually analyze and model the environment, with the objective of detecting important visuals cues and possible areas for implantation. Each student modelled and textured one model, and in a collaborative effort the models were joined together to obtain the model of the whole square.
In the next step, the students adapted a work by Victor Vasarely (1906-1997), member of the Opt-Art movement and author of paintings, murals and sculptures for public spaces such as the Central University of Venezuela in Caracas. After studying his works, each student took a painting as a reference for creating a sculptural work.
The next phase consisted in modelling the sculpture proposals to be inserted in the square (using AutoCAD©, Rhino© or SketchUp©). The sculptural shapes suggested by the paintings had a variety of complex forms and each case had to be approached by students differently.
The final step was the initial urban design proposal, where the scale and location of the different elements of street furniture, pavement and sculptural elements were defined after studying the virtual model and visiting the site. After this first proposal, each student produced a couple of non-photorealistic sketches (Figure 6).

Content preparation for AR
The group using AR technology produced the necessary 3D content in the classroom with the assistance of the lecturers:
The result (Figure 7) was a mobile application that allowed viewing the student- created virtual models overlaid on the real environment using the mobile device camera, with the capacity to rotate and scale the 3D model of the sculpture using the touchscreen.
In the specific case of the Android platform, the students were able to define several information channels, consisting of different 3D models associated with markers. The resulting application was capable of switching between the sculptures when a marker was recognized by the device camera.

On-site verification of proposals using AR and collaborative ML (test group only)
Once the students had created the content and the applications, they visited the site location to visualize their proposals and those of their peers in the real context. This peer-review could be conducted on iOS devices by physically exchanging devices, while on Android the application was able to display the content uploaded on Moodle.
With the AR applications, the students were able to position their proposal on the real environment viewed through the camera in their devices, and examine the proposals of their fellow students from different points of view. As a result of that interaction, the students were able to discuss the proposals made by each member of the group, and adjust the scale of their sculptural model to better fit the environment.
In addition, students were asked to assess two different visual styles: a realistically shaded model and a NPR shaded model. The fist had a more life-like appearance, incorporating lighting and shadowing, but required more effort to produce (using the ARmedia plugin for 3DStudio) and resulted in models with needed more hardware resources, while the second did not require texturing and rendering and could be more easily produced using SketchUp.
Final presentation of the project using non-AR other technologies and post-course survey
For the final presentation of the proposals, all the students resorted to the technologies used in previous editions of the course, producing day and night scenes (Figure 8) using 3D rendering packages, using the educational versions of Artlantis (22 students) and 3DStudio (3 students). When asked for the reason of their choice, the majority of students chose Artlantis for its perceived ease of use, good interoperability with SchetchUp and quality of results. The results were processed using digital image editing applications to equalize the contrast of different elements and composite backgrounds, characters and vegetation to denote a better sense of scale and atmosphere.

In a post-course survey, the students were asked to evaluate their experience in the course regarding the usability of the applications, the devices and the satisfaction with the educational experience. The AR group was also asked about the use of AR in education, as well as their opinion on the use of AR in the course, their perceived relation between 3D modeling and AR and suggestions for improving the course.
All surveys were conducted through Google Forms© and the users’ assessment through questionnaires was based on ISO-9240-11, which provides usability guidelines based on three main variables: effectiveness, efficiency and degree of satisfaction.
Experimental results
Prior to the statistical analysis, so as to verify that the groups were comparable (Table 1) and did not have significant differences that would make the final results inconsistent, the variation in the ratings between different groups was checked to ensure it was not significantly greater than the variation within the groups.
| Group | N | Average | Std. Deviation |
| AR on Android | 8 | 6.120000 | 1.7154092 |
| AR on iOS | 9 | 6.297778 | 2.0043127 |
| Not using AR (control) | 8 | 6.166250 | 2.2128710 |
| Total | 25 | 6.198800 | 1.9056983 |
The null hypothesis that the grades of the members of the different groups overlap (that is, there is no significant difference) was tested using Analysis of Variance (ANOVA). The analysis of the qualifications before the beginning of the course among the three groups showed no significant difference (Table 2) between the groups in terms of their qualifications before conducting the training (F2,22 = 0.018, p = 0.982).
| Sum of Squares | df | Average Square | F | Sig. | |
| Between Groups | .146 | 2 | .073 | .018 | .982 |
| Within Groups | 87.014 | 22 | 3.955 | ||
| Total | 87.160 | 24 |
From ANOVA assessment for PRE marks, we can conclude that there are not differences between groups before the experiment (F2, 22=0.018, p-valor=0.982). All students showed similar marks before doing the course.
Despite these initial similarities, when comparing the students that used AR technology in their architectural proposal to those who did not, the results showed that AR-using students significantly increased their scores compared to the conventional control group (Table 3).
| Group | Statistic | Pre-Course | Post-Course | Improvement |
| AR on Android | Average | 6.120000 | 7.511250 | 1.3912 |
| Std. Deviation | 1.7154092 | .9918588 | 1.45596 | |
| AR on iOS | Average | 6.297778 | 7.861111 | 1.5633 |
| Std. Deviation | 2.0043127 | 1.0982537 | 2.00806 | |
| Not using AR (control) | Average | 6.166250 | 6.076250 | -.0900 |
| Std. Deviation | 2.2128710 | 2.0716794 | 3.63077 | |
| Total | Average | 6.198800 | 7.178000 | .9792 |
| Std. Deviation | 1.9056983 | 1.5991899 | 2.52446 |
To test the improvement in grades of the students in the group using AR technology, the Student T-Test was used because it was considered more appropriate for small samples. The null hypothesis was that there was no difference in grades when using AR technology compared to the control group following the traditional course.
The results (Table 4) showed that the experimental groups (Android and iOS) had significant differences between the average grades before and after the course (sig.<0.05), and therefore the null hypothesis was rejected for these groups. However,the grades did not change significantly for the control group, and therefore in thiscase the null hypothesis was accepted.
| Pair (before and after course) | Mean Diff. | Lower 95% CI | Upper 95% CI | t | df | Sig. (2-tailed) |
| AR on Android | -1.39 | -2.60 | -.17 | -2.70 | 7 | .031 |
| AR on iOS | -2.11 | -3.89 | -.34 | -2.75 | 8 | .025 |
| Not using AR (control) | .090 | -2.94 | 3.12 | .070 | 7 | .946 |
According to the independent samples, tests (Table 5) of the three group pairs the following conclusions were extracted:
Therefore, our findings supported that using AR applications in mobile devices results in a significantly greater academic performance of the students.
| Indep. samples test | Equal Variance | F | Sig. | t | df | Sig. (2-tailed) | Mean Diff. | SE Diff. | Lower CI | Upper CI |
| Android & Control | Assumed | 3.325 | 0.090 | -1.767 | 14 | 0.099 | -1.435 | 0.8121 | -3.17 | 0.30 |
| Not assumed | -1.767 | 10.05 | 0.108 | -1.435 | 0.8121 | -3.24 | 0.37 | |||
| iOS & Control | Assumed | 2.954 | 0.106 | -2.258 | 15 | 0.039 | -1.78 | 0.79 | -3.47 | -0.10 |
| Not assumed | -2.180 | 10.37 | 0.053 | -1.78 | 0.82 | -3.60 | 0.031 | |||
| Android & iOS | Assumed | 0.041 | 0.843 | 0.686 | 15 | 0.503 | .0349 | 0.51018 | -0.73 | 1.43 |
| Not assumed | 0.690 | 14.99 | 0.501 | .0349 | 0.50694 | -0.73 | 1.43 |
Student satisfaction using AR technology in the course
As we mentioned before, we evaluated the usability using questionnaires based on ISO 9241-11, which provides usability guidelines (effectiveness, efficiency and satisfaction). Responses average were very similar, ranged from 3.59 to 3.73, out of 5 (see Figure 9).

The overall assessment of the courses was rated 4.18 points out of 5. This gives an idea of the degree of satisfaction achieved. In a correlation analysis between the course global opinion and the other variables, a high correlation (0.68) was detected with the representativeness of the exercise. Therefore, this variable seems crucial to the success of this kind of teaching experience. Global opinion was not so correlated with the fact of being able to solve the exercises independently. The strongest correlation (0.89), however, was with the obtained gain variable. Students with little gain between pre and post qualifications rated worse than student from experimental groups (with higher gains). On other hand, variables related to prior knowledge of technology and to the use of different software and operating systems did not correlate significantly with the course global opinion.
CONCLUSIONS
The educational experience described in this paper tested the applicability of AR in educational contexts in the fields of architecture and urban planning and design. The research question was whether putting AR technology in the hands of students (both as consumers and producers of content) improved their academic performance. The experimental results of a comparison between a test group (students using AR) and a control group (students following a conventional course) showed that while the grades of the two groups were similar at the beginning of the course, there was a significant improvement on the grades of the former group, and not on the control group, supporting the hypothesis that AR technology is a valuable educational tool in the architecture and urban planning and design fields. We can conclude that the first advantage of the use of Mobile Learning in a learning environment is the interest that has aroused in all students and the fact that by using this type of technologies they can upgrade the classical photomontages, together with the possibilities that AR allows them to present and share their projects.
The experimental group was able to:
However, we must be aware that this technology needs more technical development, flexibility and stability in their applications. Its use in teaching requires certain knowledge that in many cases the teacher and the students are not willing to acquire. For example, the markers used had to be placed within 10-15m of the users due to technical limitations imposed by the resolution of the cameras in the phones at the time of the experiment. Since the marker had to be always present in the frame, this created problems because it was not possible to get away from the marker so as to get a wider perspective of the environment. Another avenue of research will imply improving the realism of the objects, in aspects such as believable lighting and shadowing that matches the environment, and increasing responsiveness (frames per second). Besides, the most obvious result is that the size of sculptures using AR visualization should not exceed 2.5 meters in height to be adjusted, and that with few elements the place already looks full.
In addition, the students expressed evident satisfaction and motivation regarding AR technology in the survey following the course. Some advantages have been found over other commercial applications such as the possibility to display several 3Dmodels without changing the marker (which allows to compare different architectural proposals and hypotheses), and the ability to move objects in the scene. These benefits were crucial to the viability of the study outdoors, helping to minimize the problem of stability in the scene of that kind of AR systems, based on optical recognition.
Future research
We found significant differences in academic results depending on which of the two teaching scenarios were used, and results show significant differences in student satisfaction and motivation. However, similar experiences with larger groups must be repeated to compare these results. When improving this need we have developed other parallel educational experiments working with AR and VR that confirm the results of the project explained in this article (Sánchez, Redondo, Fonseca, & Navarro, 2014; Valls, Redondo, & Fonseca, 2015; Navarro, De Reina, Rodiera, & Fonseca, 2016). Results suggest that the combination of an attractive technology and the user-machine interaction that AR entails make students feel more motivated.
As a future line of work, the inclusion of mixed reality educational applications can offer a better experience in the visualization of advanced models (Mateu, Lasala, Alamán, 2014) promoting inclusive education of students with disabilities. These methods can also be used in public participation processes in the development of urban proposals beyond architecture education, area in which the authors are currently working (Fonseca, Valls, Redondo, & Villagrasa, 2016).
REFERENCES
Adkins, K. (2014). Aesthetics, Authenticity and the Spectacle of the Real: How Do We Educate the Visual World We Live in Today? International Journal of Art & Design Education, 33, 326-334.
Cartes, I. (1997). Art in the urban landscape, Urban Design International, 2(4), 189-198.
Burton-Chellew, M. N., Mouden, C. E., & West, S. A. (2016). Conditional cooperation and confusion in public- goods experiments. PNAS, 113, 1291-1296.,
Döllner, J., Baumann, K., & Buchholz, H. (2006). Virtual 3D city models as foundation of complex urban information spaces. In 11th International Conference on Urban Planning and Spatial Development in the Information Society. Vienna, Austria, 107-112.
Dunleavy, M., Dede, C., & Mitchell, R. (2008). Affordances and Limitations of Immersive Participatory Augmented Reality Simulations for Teaching and Learning, Journal of Science Education and Technology, 18(1), 7-22.
Erskine, M. A., Gregg, D. G., Karimi, J., & Scott, J. E. (2015). Geospatial Reasoning Ability: Definition, Measurement and Validation. International Journal of Human-Computer Interaction, 31(6),
Fonseca, D., Martí, N., Redondo, E., Navarro, I., & Sánchez, A. (2014). Relationship between student profile, tool use, participation, and academic performance with the use of Augmented Reality technology for visualized architecture models. Computers in Human Behavior, 31, 434-445.
Fonseca, D., Valls, F., Redondo, E., & Villagrasa, S. (2016). Informal interactions in 3D education: Citizenship participation and assessment of virtual urban proposals. Computers in Human Behavior, 55, 504-518.
Giesbers, B., Rienties, B., Tempelaar, D., & Gijselaers, W. (2013). Investigating the relations between motivation, tool use, participation, and performance in an e-learning course using web- videoconferencing. Computers in Human Behavior, 29(1). 285-292.
Gruen, A., Behnisch, M., & Kohler, N. (2009). Perspectives in the reality-based generation, nD modelling, and operation of buildings and building stocks. Building Research & Information, 37(5-6), 503- 519.
Haugstvedt, A. C., & Krogstie, J. (2012). Mobile augmented reality for cultural heritage: A technology acceptance study. In Mixed and Augmented Reality. 2012 IEEE International Symposium on, 247-255.
Hortet, L., & Adrià, M. (1987). VV.AA. Barcelona, espais i escultures (1982- 1986), in L. Hortet & M. Adriá, (Eds.), Fundació Joan Miró Barcelona.
Kaufmann, H., & Schmalstieg, D. (2003). Mathematics and geometry education with collaborative augmented reality. Computers & Graphics, 27(3), 339-345.
Kristoffersen S., & Ljungberg, F. (2000). Mobile Informatics. BRAA, K. et al. Planet Internet. Lund, Studentlitteratur.
Kuliga, S. F., Thrash, T., Dalton, R. C., & Hölscher, C. (2015). Virtual reality as an empirical research tool — Exploring user experience in a real building and a corresponding virtual model. Computers, Environment and Urban Systems, 54, 363-375.
Leopold, C., Górska, R. A., & Sorby, S. A. (2001). International Experiences in Developing the Spatial Visualization Abilities of Engineering Students. Journal for Geometry and Graphics 5(1), 81-91.
Martín-Gutierrez, J., Saorín, J. L., Contero, M., Alcañiz, M., Pérez-López, D. C., & Ortega, M. (2010). Design and validation of an augmented book for spatial abilities development in engineering students. Computers & Graphics, 34(1), 77-91.
Mateu, J., Lasala, M. J., & Alemán, X. (2014). VirtualTouch: a tool for developing mixed reality educational applications and an example of use for inclusive education. International Journal of Human- Computer Interaction, 30(10), 815-828.
Matsumoto, T., Hashimoto, S., & Okude, N. (2008). The embodied Web: embodied Web-services interaction with an umbrella for augmented city experiences, Computer Animation and Virtual Worlds, 19(1), 49- 66.
Monclús, F. (2010). The Barcelona model: An original formula? From ‘reconstruction’ to strategic urban projects, (1979-2004). Planning Perspectives. Ed. Taylor and Francis.
Müller, P., Zeng, G., Wonka, P., & Van Gool, L. (2007, August). Image-based procedural modeling of facades. In ACM Transactions on Graphics, 26(3), 85.
Naismith, L. (2004). Literature review in mobile technologies and learning. NESTA Futurelab series, report 11. Bristol, UK: NESTA Futurelab.
Navarro, I., Fonseca, D., Redondo, E., Sánchez, A., Martí, N., & Simón, D. (2012). Uso de la realidad aumentada como plataforma educativa en la visualización arquitectónica. En Proc. De 7ª Conferencia Ibérica De Sistemas y Tecnologías de la Información, 685-690.
Navarro, I., De Reina, O., Rodiera, A., & Fonseca, D. (2016). Indoor positioning systems: 3D virtual model visualization and design process of their assessment using mixed methods: Case study: World heritatge buildings and spatial skills for architecture students, 11º Conferencia Ibérica de Sistemas y Tecnologías de Información, Gran Canaria (Spain), En Actas CISTI, (pp. 136-142).
Park, J. Y. (2011). Design Education Online: Learning Delivery and Evaluation. International Journal of Art & Design Education, 30, 176-187.
Payne, R. (2012). Seen, Unseen or Overlooked? How can Visual Perception Develop through a Multimodal Enquiry? International Journal of Art & Design Education, 31, 245-255.
Redondo, E., Fonseca, D., Sánchez, A., Navarro, I. (2012). Augmented Reality in Architecture Degree. New Approaches in Scene Illumination and User Evaluation, Journal of Information Technology and Application in Education, 1(1), 19-27.
Sadurni, L., & Ramujkic, V. (2002). Olimpics Sculpture Guide, in E. Tolosa & D. Romaní (Eds.), Barcelona, Rotor.
Sánchez, A., Redondo, E., & Fonseca, D. (2012). Developing an augmented reality application in the framework of architecture degree. In Proceedings of the 2012 ACM workshop on User experience in e-learning and augmented technologies in education, 37-42.
Sánchez, A., Redondo, E., Fonseca, D., & Navarro, I. (2014). Academic Performance Assessment using Augmented Reality in Engineering Degree Course, 44th Annual Frontiers in Education Conference, Madrid (Spain), In Proceedings FIE, 1527-1533.
Semmo, A., Trapp, M., Kyprianidis, J. E., & Döllner, J. (2012, June). Interactive Visualization of Generalized Virtual 3D City Models using Level-of-Abstraction Transitions. In Computer Graphics Forum. 31(3), 885-894.
Sinker, R., Giannachi, G., & Carletti, L. (2013). Art Maps – Mapping the Multiple Meanings of Place. International Journal of Art & Design Education, 32, 362-373.
Sun, J., & Hsu, Y. (2013). Effect of interactivity on learner perceptions in Web-based instruction, Computers in Human Behavior, 29(1). 171-184.
Trias, E. (1976). Artista y la ciudad. Madrid: Ed. Anagrama.
Tsvetozar, G., Evgenia, G., & Smrikarov, A. (2004). M-learning - A new stage of e-learning. International Conference on Computer Systems and Technologies
Valls, F., Garcia-Almirall, P., Redondo, E., & Fonseca, D. (2014). From raw data to meaningful information: a representational approach to cadastral databases in relation to urban planning. Future Internet, 6(4), 612-639.
Valls, F., Redondo, E., & Fonseca, D. (2015). E-Learning and Serious Games: New trends in Architectural and Urban Design Education. 2nd International Conference on Learning and Collaboration Technologies. Los Angeles, USA, LNCS V, 9192, 632-643.
Wang, T. (2011). Designing for Designing: Information and Communication Technologies (ICTs) and Professional Education, International Journal of Art & Design Education, 30, 188-199.
Zlatanova, S., Itard, L., Kibria, M. S., Van & Dorst, M. (2010). A user requirements study of digital 3D models for urban renewal. Open House International, 35(3).
Notas de autor
E-mail: ernesto.redondo@upc.edu
Internationally certified by Autodesk, his activities deal with multimedia, architecture, usability studies, and computer-mediated communication. President of final year work, final year project, final bachelor’s degree and master’s degree project boards at La Salle-UPC. Member of several doctoral boards.
E-mail: fonsi@salle.url.edu
E-mail: albert.sanchez.riera@upc.edu
E-mail: inavarro@salle.url.edu