Tuesday 13 September 2011

Analysis of when peak VLE activity has occurred

Purpose

To look at the total counts from events logging and determine when peak activity occurs.

Background

Caret has developed a tool that can be run to extract total event counts by month or week from the VLE logging data. This document will explore some of the trends we have found from this data.

Warnings

The event logging data we are working with was collected until February 2011 so does not include full data for the last academic year (2010-11).

The week numbers were added to the underlying events file by using a function within MySQL. This is not ideal in that it leaves part weeks at the beginning and end of year and these points needed some manual fixing within the spreadsheet. It may also have helped to number weeks starting from the beginning of the Academic Year.

On initial analysis, a very large proportion of event activity was found to be from an event (pres.begin and pres.end) for which logging was later switched off. This has been disregarded from our counts.

Method

A script was run that produced monthly event counts from the VLE activity data over the last 5 years – this ran until Feb 2011.

The count for each event type was compared with the total count to produce a percentage. These results were then filtered to produce charts according to the scale of usage.

Results

The top 2 events were found to be reading content (green - 44% of events logged) and user logins (dark green - 22% of events logged). The chart below shows monthly counts for these events.

Content read and logins Chart
Whilst the content read shows a regular peak in Oct/Nov with a secondary peak around May, the logins do not show such a clear pattern. Both show a clear gradual underlying increase over the years. More investigation would be needed to find out exactly what is causing the peaks in logins. This is especially the case since it bears little relation to the number of user logouts recorded (see later graph).


The following shows a weekly count for content read giving more granularity of when exactly the peaks are. The main peaks are repeated in late October with secondary peaks in Feb and mid-June.

Weekly Chart

This considers weekly counts over the last year. (Feb 2010-2011).

Term dates were:

12th January - 12th March 2010. Easter (5th April 2010) shows a slight dip.

20th April - 11th June 2010. A gradual increase with a peak at the end of May.

This coincides with the examination period. There is a long dip over the summer vacation period.

5th October-3rd December 2010 First peak is early October with the highest peak at the end of October. This peak at the end of October might not be expected and it would be interesting to find out more about what is going on here. There is a large dip over the Christmas period

18th January - 18th March 2011. There is a wide peak during January and early February.

Content Read Year Chart

Although content read does have a peak at the start of the Academic Year in October, there is a higher peak in November. It would be interesting to investigate if there is an explanation for this peak half way through the term – such as a large number of students starting assignments which rely on Camtools.

New Content Chart

This shows the next active events in terms of counts. Note the scale is roughly 10% of that for viewing content and logging in. However the overall peaks for site visits, search and user logout very much mirror those for viewing connect and logins showing consistent periods of peak activity corresponding to term dates.

As one might expect, lots of new content (blue line) has been added each October, at the start of the academic year, but there is a more recent and much higher peak in January of this year.

The following series of charts show analysis of the next highest peaks broken down by event type. Note the scale is several times smaller that that for reading content.

The calendar event is logged when users create (red line) or revise (yellow line) site calendars which may be used to display lecture times, assignment submission dates and so on. There is a clear October peak for both creating and revising calenders.

Calendars Chart


Swift Chart

This tool has been added in the last few years and is used to for student surveys. The green update peaks in February would correspond to lecturers preparing surveys at this point which are returned by students roughly a month later (blue line).



Wiki Chart
The counts for Wiki activity show a peak in new material just prior to the October start of year with high revision activity at the start of October.


Site Maintenance Chart


Site Maintenance demonstrates a clear October peak with a secondary peak in Jan/Feb. There is clearly a gradual increase in the number of sites being updated (red line) whereas the number of new sites (blue line) has leveled off.


Premissions Chart

Permissions changes reflect people being added or updated on sites. Again peak activity is in October.

Forums Chart

Forums again show a peak in October but their usage looks like it has tailed off.

Chat Chart

The chat tool also shows peaks in October and for the last year in November too.


Roster Chart

Again the roster tool, which is used to view other members of a site shows peak usage in October.

One area that I was particularly asked to look at was the Tests and Quizzes (T&Q) tool. The scale for this is again of a much smaller order of magnitude but again shows October peaks:

T&Q Chart

This final chart shows the overall totals for each month (blue) and the monthly totals excluding logins and content read (orange). This again shows a peak in October and other peaks corresponding to the terms. There is not much growth, in terms of event counts, over recent years in these other activities.


Overall Chart

Conclusions

Many activities show a clear usage peak corresponding with the start of the Academic year in October. However reading content, which accounts for over 44% of the counts, has a higher peak in November than in October. It would be interesting to learn exactly what is causing this November peak.

It also seems likely, given the different pattern in logouts, that the high spikes in login activity seen in the first chart are caused by spurious data and further investigation is needed here.





2 comments:

  1. Have you tried segmenting the data to see whether you get peaks at different points in time from different courses, for example?

    ReplyDelete
  2. Tony
    Thanks for your interest and feedback.

    This is indeed something we could take further - the analysis tool that we have developed (blog to follow) allows us to extract weekly counts for each event for a given Site. We could indeed use this for busy course sites to produce weekly counts which we could chart in a similar manner.

    The first issue would be to find sites with significant activity.

    ReplyDelete