My Research Essay

Usability, requirements analysis and feasibility study, to update the UCD student portal, Blackboard, through the integration of a digital collaboration tool.

Complete Research Paper

Scully, Nicole and McEntee, Mary-Elizabeth UCD School of Business, Ireland, Nicole.scully@ucdconnect.ie and mary.mcentee@ucdconnect.ie

Abstract

The purpose of this paper is to analyse the current student portal utilised in University College Dublin and to assess how this digital product can be updated to benefit its user base. To conduct our analysis of a suitable update to the portal, we conducted both primary and secondary research. Our primary research took the form of a questionnaire, interviews and observation research. The secondary research conducted for the purpose of this paper included analysing a case study and a literature review. Our research led us to conclude that embedding a collaboration tool into the current product would make it a more user friendly and engaging site. Upon decision on the update required, a usability, requirements analysis and feasibility study were conduct to understand the scope involved in embedding such a tool.

Keywords: Student Portal, Blackboard, Collaboration, Research

1.                Introduction

Our research essay investigates Blackboard, a product which is utilised on university campuses worldwide. This digital product aims to “help learners access a safe & secure place to learn, an individualized & sophisticated way to learn, and an engaging lifelong path to their education goals” (Blackboard, 2017).

We conducted in-depth research, focusing primarily on improving the offering for students studying at University College Dublin. We did so through identifying current problems faced by the users of the student portal, understanding user needs and investigating the possible advantages associated with embedding a collaboration tool which would enable students to communicate anytime, anywhere and on any device.

Students at the University engage with Blackboard primarily to retrieve relevant course materials, to upload assessment components, and to view grades. The tool is not particularly user friendly, with many of the features being unknown and unused by its user base, however it is the only avenue for students to view such information. 

As discussed throughout our essay, we suggest that this portal be updated and enhanced, to make it a “one stop shop” where students can retrieve all necessary content and materials associated with their relevant assessment, work collaboratively with their team members to complete the required activity, and then upload the finished product for grading - all without needing to leave the student portal.

2.                UCD Blackboard and Its Initial User Requirements

The UCD Blackboard portal was originally built to facilitate the following useful functions:

·  The distribution of course-relevant materials including PowerPoint slides, readings etc., necessary for university course modules.

·   Enable students to upload components of continuous assessment assignments, both removing the need to print physical copies of their coursework, and the need to be on campus to submit the assignment.

·    View received grades.

·   Communicate with academic staff and peers through announcements and discussion boards.

3.                Problem

To make an informed update to the UCD student portal, we first aimed to understand the problem at hand to ensure our solution would best address the needs of the students of University College Dublin.

The version of Blackboard which is currently utilised is a relatively static site; a site by which activity is limited to an administrator uploading information to a destination point, and a user retrieving said information. Beyond that, there is limited functionality pertaining to content creation, user interaction, and there are very evident gaps between the portal objectives and realistic practices.

Group-oriented assessment has been central to our third level education to date, and it is an integral course component in University College Dublin. Given the deeply collaborative nature of this style of working, the fact that the current version of Blackboard, the official University-endorsed portal, does not provide any form of collaborative functionality appears to be a significant misalignment between the proposed objectives of group work, and the realistic outputs achievable.

4.                Need

Upon uncovering the problems associated with the UCD student portal, Blackboard, we aimed to discover the needs of those engaging with the site.  To do this effectively, we decided to conduct a questionnaire using Survey Monkey, of a segment of the student populace. Our choice to use a questionnaire as the means to discover the needs of users stemmed from the fact that we could reach out to a large base of students with relative ease, whilst ensuring anonymity to its recipients.

We identified from our questionnaire that users of Blackboard required a more engaging site; to them, the current product had not been updated to incorporate Web 2.0 features, such as content curation and collaboration.

The clear outcome of this initial round of research was that the students of UCD needed a more interactive student portal; one which enabled them to communicate with their peers and academics. In order to create an appropriate solution to meet these needs, we carried out additional research, described in detail further below.

5.                Research Methods

To confirm that our proposed solution would adhere to the needs of the portal users, we decided to employ a variety of research methods to achieve a detailed view of the current user features, and how these could best be improved upon to update the digital product.

The research methods adopted for this report included both primary and secondary research. In terms of primary research, we conducted a questionnaire and a “fly on the wall” approach, a means of exploratory primary research. Exploratory research is useful as it can help “identify problems” (DeVault, 2017).

We subsequently conducted specific primary research which is used to solve problems “discovered through exploratory research” (DeVault, 2017). Our specific primary research involved interviews with a current student, a recent graduate, and faculty members, within the University. Once we established the solution to update the student portal, we carried out an interview with a Product Manager at Skype to uncover more about collaborative tools, and their place in an educational context. The findings of this interview can be found in the appendix section of this paper.

To compliment and support our primary research, we conducted secondary research, through examining a case study of the implementation of Blackboard Collaborate, a product developed by the creators of the UCD student portal. Finally, we analysed existing literature on the topic of collaborative tool usage in an educational context.

To further explore the topic of research, we conducted a usability study of our proposed solution, a collaborative tool, as well as on the Blackboard platform itself, in order to determine the feasibility of the proposed improvements. In doing so, we identified numerous pain points in Blackboard’s existing user journey and realised that in order to have a seamless upgrade, we would need to address how the user interacts with the platform before implementing additional steps in their journey. In addition to the web platform, the mobile app was an avenue we looked at in terms of incorporating collaborative tools; potentially enabling the user to access group chats, conference calls, shared information and files, at any time, in any place. Given the short time-frame associated with this project, we feel that the mobile app development should be revisited in greater detail at a later stage.

6.                Primary Research

6.1.            Questionnaire

The first research method we utilised was a questionnaire, which we conducted using Survey Monkey. We received 91 responses from individuals who were either faculty members, current University students or recent graduates. Our motivation behind using this research method was to confirm our hypothesis, that collaboration is an intrinsic aspect of third level education and that students would interact and utilise the student portal, Blackboard, on a more frequent basis, if a collaboration tool was to be built into its feature set. We distributed this survey to a wide audience to try and facilitate varied feedback.

Our questionnaire reinforced our initial beliefs that there is an increasing emphasis on group assessment in universities. Of the 91 respondents, 93.41% stated that their course either involves, or involved, a degree of group assessment, or teamwork.

The questionnaire mirrored our own experiences with group assessment, in that 76.67% stated it was challenging to organise in-person group meetings, with over 75% resorting to collaborative tools such as Google Docs, to complete the relevant components of their assignments. The most notable findings from our survey concluded that there is in fact a great desire for an official University collaboration tool. Another interesting observation arising from this research was that, 86.67% of respondents indicated that an official University digital collaboration tool enabling IM, file sharing, group conferencing etc., would better facilitate the group assessment process.

As previously stated, our primary motivation was to obtain as much feedback through the questionnaire as possible. For this reason, we made the decision to extend the scope of our target audience, to include a number of universities in various geographic regions. However, a potential pitfall associated with this research method centres around the fact that the study was taken by students and graduates outside University College Dublin, therefore taken by individuals who had not previously engaged with the UCD Blackboard portal. Despite this, our findings gave us conviction in our proposed solution and enabled us to proceed with additional research.

Detailed results of this survey can be found in the table below and the appendix section of this research essay.

Question	Yes	No
Did / Does Your Course Involve Group Assessment / Teamwork	93.41%	6.59%
Was Organizing In-Person Group Meetings Challenging?	76.67%	23.33%
Did You / Do You Use Collaboration Tools (e.g. Google Docs) to Complete Group Projects	75%	25%
Would an Official University Online Collaboration Tool Enabling Instant Messaging / Group Conferencing Calls Have Made / Make Team Work Easier?	86.81%	13.19%
If You Were Unable to Physically Attend a Lecture, Would Joining Online / Listening Back Interest / Have Interested You?	94.51%	5.49%

Table 1.   This table outlines some of the key insights which arose from conducting the questionnaire

6.2.            Interviews

Further research of this topic called for a more in-depth qualitative method to collect a more concrete data set. This research took form in the conducting of interviews with individuals who are active users of the Blackboard platform; current students, graduates, as well as faculty members within the University.

We interviewed individuals with experience utilising the UCD student portal, in order to evaluate how users feel about the current product, its features, and in what ways they felt the product could be improved. From a usability perspective, we found it interesting to ask how graduates found the transition from an academic context to that of a corporate context; was their previous use of collaboration tools an advantage to them as they adapted to corporate operations, or was their lack of exposure to these tools a disadvantage to them, having to familiarise themselves with a new system of working, once they entered employment.

6.2.1.        Interview with a Current Part-Time University Student in Full-Time Employment

We conducted the first of our interviews with a current University student, studying part-time in conjunction with his full-time employment at a large multi-national software company.

The student felt that Blackboard could be improved, particularly its mobile application. He believed it fulfilled basic needs, in terms of retrieving information and submitting assessment components, but felt it was difficult to navigate, not engaging and not very intuitive. In comparison to his company portal, he thought it could be improved greatly to make it a more engaging and aesthetically pleasing site.

The interviewee is required to conduct group assessment on a regular basis as part of his studies. He explained that he finds it difficult to meet his peers in-person due to his work commitments, particularly as he is frequently required to travel with work. This has often led to a ‘divide and conquer’ approach to assignments, as opposed to maximising the benefits of working as a team. 

He has found collaboration tools such as Google Docs and WhatsApp audio a great aid to be able to complete team work, particularly when he is travelling and unable to meet in person.

6.2.2.        Interview with a Recent UCD Graduate

Our second interview was conducted with a recent UCD graduate who is now working in the Management Consulting profession. He currently works on a cross-border project with teams in India, the USA and Singapore, and so the use of collaboration tools is paramount to carrying out his role effectively.

The interviewee, who studied Business and Law at University College Dublin explained that group assessment was a key component of his Undergraduate degree. He found it challenging to work as part of a team due to team members having conflicting schedules and juggling part-time employment alongside their studies.

The interviewee stressed that although he thought use of collaboration tools would have aided his ability to complete teamwork in University, using such tools is relatively easy and frictionless and so he experienced little difficulty when required to use online collaboration tools in the workplace. This led us to consider ease of use when investigating suitable collaboration tools.

6.2.3.        Interview with a Former Tutor at University College Dublin

Our motivation behind interviewing a former faculty member within the University was to understand whether the staff perceived there to be any problems with the current Blackboard portal from a teaching standpoint, as well as to gather recommendations on how the portal can be improved to make it a more usable and engaging product for its users.

The former tutor strongly believed that the current student portal, Blackboard, left a lot to be desired. He stated that the site had poor usability and design. He also explained that the site was relatively static, comparing it to that of a governmental agency website.

He agreed that the digital product could be greatly improved through either making it more interactive or improving the design to make it more intuitive.

When we suggested our proposal to introduce a collaboration tool onto the portal, he thought it was a fantastic idea and would complement the current offering. He believed it would make Blackboard a “one stop shop”, whereby students could retrieve materials relevant to conduct their assessment, collaborate on the necessary tasks and then upload the completed assignment all without leaving the portal.

6.2.4.        Interview with a Member of the Smurfit Programme Office

The last interview we conducted was with an individual from the Smurfit School of Business programme office. Our reasoning behind this was the fact that the programme office has access to a multitude of students from a variety of backgrounds and so, could give us an accurate view of how students perceive the current offering. 

The interviewee believed that Blackboard was outdated and not aesthetically pleasing. She stressed that “if the look and feel of Blackboard was updated to reflect the current websites it may be more conducive to student engagement”.

When asked if she thought the students of her college, the Smurfit School of Business could benefit from the use of an official University collaboration tool embedded into blackboard, she answered yes and explained that “there are a lot of various tools being used but none in one specific area”.

6.3.            Fly on the Wall Research

Our “fly on the wall”, or observational research, involved assessing an individual’s use and engagement with the Blackboard portal while conducting an assignment involving group work. To obtain the relevant content pertaining to his assignment, the user journey consisted of eight clicks, or touch points, from the UCD landing page to the desired destination.

The individual’s lecturer also utilised a third-party blog platform, which contained additional content relating to the assessment component which required the student to navigate between both sites. In terms of completing group work, the student leveraged Google Docs, to collaborate with the other members of his group when in-person meetings were not possible, as the Google Docs offering enables “concurrent online editing and collaboration for knowledge building by multiple users” (Kai-Wai Chu et al. 2011).

Completing observational research enabled us to understand and evaluate the usability issues pertaining to the Blackboard student portal, as well as furthering our research as to how it could be updated to make it a more user friendly, collaborative, space.

Process map 1. This process map represents the journey the individual pursued to retrieve relevant materials during the observational research.

7.                Secondary Research

7.1.            Literature Review

A research article which aided greatly in terms of understanding the topic of collaboration and team assessment in a digital age was written in 2010 by Samuel Kai-Wai Chu and David M. Kennedy. The authors, who were interested in Web 2.0 collaboration tools, conducted a study to understand if and how these tools can aid students in group work and collaboration. They conducted research on two such tools (Google Docs and Media Wiki), with undergraduate students from an Information Management Programme at the University of Hong Kong being the research subjects. This paper aided us greatly in deciding which forms of primary research may be most significant to conduct.

The paper reiterated our hypothesis that there was a growing user need for tools to aid group assessment in universities, given their prevalent nature in current course syllabi. The authors explain that "in recent years education has been undergoing a shift from teacher-centered and instruction-based curriculum implementation to student-centered and inquiry-based learning", and that students are "routinely required to engage in collaborative learning activities such as group projects, presentations, group discussions and peer evaluations that require significant collaboration and communication with classmates” (Kai-Wai Chu et al. 2011).

The authors stress that in the global economy "collaboration has become an essential skill necessary for effective functioning in society" (Kai-Wai Chu et al. 2011), which aided us in coming to the realisation that using such tools in a university environment, in addition to facilitating the learning process, could better prepare students for employment.

7.2.            Case Study

While researching potential solutions to update the current UCD student portal, Blackboard, we were surprised to uncover that there was in fact a collaboration tool built by the same creators of the Blackboard product. We examined a case study in which the tool had been introduced into the University of Bradford.

The tool in question, Blackboard Collaborate, is a “web conferencing tool that offers a more collaborative, interactive and mobile learning experience” (University of Bradford Making Knowledge Work, Unknown).

The case study analyses the views of academics using Blackboard Collaborate, in addition to the manners in which they use it. Those interviewed as part of this case study report how the tool has enabled them to reduce business travel and allowed for intuitive cross-border collaboration.

Through studying the case, we discovered that faculty members were also able to record lectures using the product, making it accessible for those unable to physically attend lectures, as well as aiding distance learning courses.

Although Blackboard Collaborate acts as an independent product, rather than an add-on to the existing Blackboard portal, we felt that a similar solution, with functionality including lecture capture, voice audio, and IM features, could be of benefit, as an upgrade to the existing Blackboard product.

8.                Proposed Solution

Upon uncovering the problems associated with the current student portal and gaining an understanding into the needs of those using the portal, we have concluded that the product can be updated to create a “one stop shop” for students to retrieve content relevant to conduct assessment, collaborate with their team members and submit the assessment component for grading.

Our research indicated that if a collaboration tool was incorporated into the current Blackboard feature set, it would make the product more accessible, as well as give the user more incentive to use it on a frequent basis as opposed to current behaviour which sees many students only utilising the portal approaching the exam period or out of sheer necessity. This is important given that “students are routinely required to engage in collaborative learning activities such as group projects, presentations, group discussions, and peer evaluations that require significant collaboration and communication with classmates” (Kai-Wai Chu et al. 2011).

Although lecturers place a strong emphasis on group assessment and team work, students lack the training and toolset to effectively collaborate with their team members. As discussed above, our primary user base of students studying at University College Dublin require a collaborative tool which can aid them in completing group assessment competently and we concluded that a natural fit to fulfil this need would be the introduction of a collaboration tool into the UCD Blackboard portal.

UCD aims to equip its students with the skills necessary to succeed in the workplace. An example of this can be seen in the Lochlann Quinn School of Business at University College Dublin. Tutorial rooms have been designed to resemble workspaces, Bloomberg terminals have been installed with training sessions organised on a regular basis and meeting spaces with recording facilities and large monitors are available to book using the online student portal, SIS web. In the digital age, the evolution of technology has equipped us with the skill-set and tools to communicate in new and more efficient ways. Where cross-continent communication was once a near impossible task, collaborative tools, social media, and instant messaging have transformed the communication process. In a corporate environment, these tools have resulted in individuals being able to work remotely, and still efficiently align with team objectives and goals, attend meetings and maintain a steady workflow.

9.                Usability

Jakob Nielsen claimed that there were ten general principles for user interface design (Nielsen, 1994):

1.       Visibility of system status

2.       Match between system and the real world

3.       User control and freedom

4.       Consistency and standards

5.       Error prevention

6.       Recognition rather than recall

7.       Flexibility and efficiency of use

8.       Aesthetic and minimalist design

9.       Help users recognise, diagnose and recover from errors

10.   Help and documentation

In our initial phase of research, we assessed the need (or want) for an official, university-endorsed collaborative tool to be embedded into the existing university platform infrastructure - but in doing so, we uncovered a series of usability issues with the platform itself. Blackboard as an entity already has the capabilities of collaboration programmed within it in the form of a module widget, but in order to be able to use this feature, training would be required. We decided to undertake a usability study of the Blackboard platform, as well as that of a series of collaborative tools already on the market, in order to come to a conclusion as to whether or not the proposed improvement would be feasible.

With regard to the above, Blackboard meets very few of Nielsen’s general principles for user interface design. In order to rectify this, we integrated a qualitative research study into that of a usability study to pinpoint the main areas of difficulty with regard to use, and let the interview subjects think aloud as to how they’d navigate the platform with more ease. From here, we took our findings and combined them with our initial proposal for integration/improvement in order to tailor the solution to the specific needs of the prospective users.

As mentioned throughout the report, the initial inspiration for this particular area of study came from personal pain points associated with the lack of functionality of the Blackboard platform, how there were certain ways in which it could better aid the student learning experience, such as acting as a central hub for group communication, document sharing/holding, and facilitating team meetings. This personal experience acted as a form of self-assessment of the usability of the platform, which we then compared and contrasted with the assessments carried out by the interview subjects in order to draw common issues/difficulties and perhaps uncover ones that we had not previously considered in our scope.

From this point, we invited interview subjects to evaluate the usability of Blackboard in its current form, commenting on their experiences whilst using it in the process. The interviews were semi-structured, in that the questions posed were designed to provoke deep thought and consideration as to what exactly the user was doing and why they were doing it, what they desired to do and what was potentially preventing them from doing so. In saying that, the questions were also designed in a manner that eliminated any bias or preconceived notion of the platform that could have been inferred by the interviewer(s). Once the user had evaluated the platform itself, we posed open questions about collaborative technologies, which ones the user had come in contact with, which they preferred, which they found the most conducive to efficient communication and knowledge sharing and/or project management, and how they feel a collaborative tool would aid the user experience when navigating through Blackboard.

10.            Requirements Analysis

With regard to the requirements analysis portion of our research, we decided to undertake a Scenario-Based Requirements Approach. Upon referencing a report written by Allistair Sutcliffe for the School of Informatics in City University, London, where he defines scenarios as 'facts describing an existing system and its environment including the behaviour of agents and sufficient context information to allow discovery and validation of system requirements’ and states that scenarios are instances of actual experience with a system captured from users, we deduced that this particular method of requirements analysis was suitable for our area of study (Sutcliffe, 1998).

As mentioned above, the usability study shed a lot of light on the problems facing the user when using the Blackboard platform, which in turn led to the uncovering of requirements users felt would enhance or improve the product, as well as requirements necessary to support the introduction of a collaboration tool into Blackboard. What we discovered was a need for consistency, and a single, uniform hub which would hold all information, be it assignment or study-related, or a way to access everyone in participating in the course easily and efficiently.

11.            Feasibility Study

Both the usability study and requirements analysis sections of our research highly informed the path the feasibility study took.

Given that a feasibility study is designed to assess the viability of a project, both from an investment and technical capability perspective, the first step of this analysis came in the form of plotting out the perceived user needs, and subsequently conducting research to identify the actual needs of the users. In doing this, we would be able to assess the volume of work necessary to improve the usability of the Blackboard platform, which would then enable us to justifiably propose an end solution.

As part of the feasibility study, we also evaluated the compatibility of Blackboard as a host platform for a collaborative tool integration. As is further detailed in the interview segment of the appendices, two employees from a large telecommunications application software company confirmed that collaborative tools such as video conferencing software, instant messaging services and data sharing/collection points could be seamlessly integrated into educational portals. In addition, we were also granted top-level access to both quantitative and qualitative data obtained by said company through use of surveys and user focus groups as part of their research during various product updates and new feature roll-outs. This data, combined with that obtained from both our personal experiences and self-assessment of using collaborative tools, as well as that of the interviewees, gave us a much broader understanding of the perceived benefits of integrating digital collaboration tools into online portals - which was what we set out to evaluate in the first place.

With regard to a project trajectory, our recommendation for Blackboard’s upgrade comes in two steps. Given that many third-party collaborative tools are priced on a ‘per user’ basis to a certain extent, it would not be economically viable for UCD to invest. We therefore recommend that for the first two years of the project roll out, digital tools that are freely available such as the Google suite and Slack are integrated into the portal’s immediate feature-set, the only primary cost being the time and resources provided by the UCD product/tech team. Added benefits of this proposed solution are exposing the students/users to said tools, employing a sense of discipline around using them to fulfil academic needs, as well as bringing to light any areas in which training may need to be provided. Whilst this is a proposed short-term solution to a prevalent problem within the university, we also propose that on a five-year trajectory, Blackboard Collaborate’s entire suite of tools and programmes be integrated into Blackboard. Given the higher price point and larger volume of software integration requirements, we feel as though the university could benefit from a longer-term implementation trajectory, as well as its users becoming familiar with a similar suite of products and providing iterative feedback in order to maximise the performance of the permanent upgrade or solution.

12.            Conclusion

Our research has brought us to the conclusion that the UCD student portal, Blackboard, should be updated to make it a more user-friendly, engaging and collaborative space. As detailed throughout this paper, the research we conducted confirmed our hypothesis that that there is a use-case for integrating a collaborative tool into the Blackboard portal.

We recommend that the University work to incorporate a suite of collaboration tools including file sharing, video-conferencing and instant messaging to enable students to communicate with their peers and academic staff at the University. The goal behind embedding such a tool is to make it a one-stop-shop for students to retrieve relevant information relating to their assessment component, to collaborate with the team members and to upload the finished product, all without the requirement to leave the portal.

13.            References

Brad.ac.uk. (n.d.). Virtual Meetings Case Studies. [online] Available at: https://www.brad.ac.uk/smart-            travel/media/estatesandfacilities/allfiles/documents/v-m-case-studies.pdf [Accessed 19 Nov.   2017].

DeVault, G. (2017, February 2). Difference Between Primary Research and Secondary Research. Retrieved from The Balance: https://www.thebalance.com/differences-primary-and-secondary-research-2296908

Jian, Shen. Scenario Based Requirement Analysis Modeling and Design Evaluation for Real-Time Re-    active Systems. (2004). Masters Degree. Concordia University.

Nielsen, J., and Molich, R. (1990). Heuristic evaluation of user interfaces, Proc. ACM CHI'90 Conf.

            (Seattle, WA, 1-5 April), 249-256.

Nielsen, J. (1994). Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.), Usability Inspection

            Methods, John Wiley & Sons, New York, NY.

Samuel Kai-Wai Chu, D. M. (2010). Using Online Collaborative Tools for Groups to Co-construct Knowledge. Online Information Review, 581-597.

Sutcliffe, A. (1998). Scenario-based requirements analysis. Requirements Engineering, 3(1), pp.48-65.