|
|
Information and Communication Technology (ICT) is a
multi-trillion dollar industry. It offers potential for significant
organisational improvement and competitive advantage. However, ICT investment
does not always translate into monetary rewards. Reports of project failure,
budget and timescale overruns, and limited or negative returns are not
uncommon. ICT evaluation is important in determining and improving an investment’s
performance and in enabling corrective measures to be taken. However,
organisations often fail to formally evaluate their ICT systems. The
evaluation process is underpinned by numerous complexities such as
organisational politics, intangible system impacts, and limitations in
existing evaluation techniques, and ICT evaluation remains a topic of ongoing
debate. While ICT evaluation research is active across many business sectors,
a paucity of evaluation research is apparent in Higher Education.
This study examines ICT evaluation in the Higher
Education sector. It evaluates ex-post the implementation of a large-scale
student MIS in the Irish Institutes of Technology (IoTs). This system, which
was rolled out to 15 Institutions, sought to standardise student
administrative operations within all of the IoTs. System operationalisation
proved complex and many problems were initially experienced. In particular,
system functional capabilities and IoT operational requirements were
misaligned. Despite these difficulties, several benefits also materialised.
At the time of research, the full system impact was not understood as the ICT
investment had not been formally evaluated.
Within this research study, case-studies based on
multiple sources of evidence were carried out in five IoTs. Evidence
collected from 49 semi-structured interviews, project documentation and
reports provided in-depth insights into the key issues associated with
operationalising the MIS. The evidence was analysed using a grounded theory
method and the process was supported by qualitative data analysis software.
The key concepts that emerged from the evidence were reported in a cross-case
primary narrative. An in-depth examination of this primary narrative resulted
in the distillation of 15 findings which provided insights on five different
aspects of the project - system selection, system development, system commissioning, system
ex-post performance in the early years and system ex-post performance at the
time of research. Through a process of reflection, these findings were
consolidated into a new theoretical contribution. This emphasised the
importance of continuous participative evaluation in addressing misalignment
between a standard system’s functional capabilities and the diverse operational
requirements of multiple autonomous organisations. A model and set of
guidelines for resolving functional-operational misalignment was also
developed. Finally, the study’s limitations and avenues of further
research were identified.
|
Marian Carcary
|
|
