FIND A SOLUTION AT Academic Writers Bay
Independent assurance report to Parliament2017–18: 22The VictorianGovernment ICTDashboardJune 2018
The VictorianGovernment ICTDashboardIndependent assurance report to ParliamentOrdered to be publishedVICTORIAN GOVERNMENT PRINTERJune 2018PP no 398, Session 2014–18This report is printed on Monza Recycled paper. Monza Recycled is certified Carbon Neutral by the Carbon Reduction Institute(CRI) in accordance with the global Greenhouse Gas Protocol and ISO 14040 framework. The Lifecycle Analysis for MonzaRecycled is cradle to grave including Scopes 1, 2 and 3. It has FSC Mix Certification combined with 55% recycled content.ISBN 978 1 925678 24 6The Hon Bruce Atkinson MLC The Hon Colin Brooks MPPresident SpeakerLegislative Council Legislative AssemblyParliament House Parliament HouseMelbourne MelbourneDear Presiding OfficersUnder the provisions of section 16AB of the Audit Act 1994, I transmit my reportThe Victorian Government ICT Dashboard.Yours faithfullyDave BarryActing Auditor-General20 June 2018
Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 5ContentsAudit overview ………………………………………………………………………………………………. 7Conclusion ……………………………………………………………………………………………………………………….. 8Findings ……………………………………………………………………………………………………………………………. 9Recommendations …………………………………………………………………………………………………………… 12Responses to recommendations ……………………………………………………………………………………….. 131 Audit context …………………………………………………………………………………………… 151.1 Background …………………………………………………………………………………………………………….. 151.2 The Victorian Government ICT Dashboard ………………………………………………………………….. 211.3 Legislation and standards …………………………………………………………………………………………. 261.4 Governance framework ……………………………………………………………………………………………. 271.5 Agency roles …………………………………………………………………………………………………………… 281.6 Previous audits ……………………………………………………………………………………………………….. 291.7 Why this audit is important ………………………………………………………………………………………. 291.8 What this audit examined and how …………………………………………………………………………… 291.9 Report structure ……………………………………………………………………………………………………… 302 The ICT Dashboard—transparency and oversight ………………………………………… 312.1 Conclusion ……………………………………………………………………………………………………………… 322.2 Has the dashboard improved transparency? ………………………………………………………………. 322.3 Is it easy to understand and accessible? …………………………………………………………………….. 332.4 Is the information timely and complete? ……………………………………………………………………. 342.5 Does it provide better oversight? ……………………………………………………………………………….363 The ICT Dashboard—agency data and reporting ………………………………………….. 393.1 Conclusion ……………………………………………………………………………………………………………… 393.2 Are agencies following the ICT Reporting Standard? ……………………………………………………. 403.3 Are agencies reporting accurate data? ……………………………………………………………………….. 443.4 Do agencies consistently apply the guidance? …………………………………………………………….. 483.5 Are agency systems and processes adequate? ……………………………………………………………. 52Appendix A. Audit Act 1994 section 16—submissions and comments ………………… 55Appendix B. Detailed results from agency testing …………………………………………….. 67Appendix C. ICT project reporting requirements ………………………………………………. 716 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportAcronyms
Business as usualChief Financial Officer
Chief Information OfficerDepartment of Economic Development, Jobs, Transport and ResourcesDepartment of Health and Human ServicesDepartment of Premier and CabinetDepartment of Treasury and Finance
Financial Management Act 1994Financial Reporting Direction
High Value High RiskInformation and communications technologyInformation technologyMelbourne WaterProject management officePublic reportingPublic Transport VictoriaRed/amber/greenUnited States of AmericaVictorian Auditor‐General’s OfficeVictorian Agency for Health InformationVictorian Secretaries’ Board
Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 7Audit overviewInformation and communications technology (ICT) is integral to howgovernments manage information and deliver programs and services. ICTprojects need to be diligently monitored and successfully implemented, sothat services to the government, public sector and community can be efficientand effective.Comprehensive reporting on ICT expenditure and projects is important toimprove transparency and provide assurance that public sector agencies haveused public resources in an efficient, effective and economic way.Our previous 2015 auditIn our 2015 report Digital Dashboard: Status Review of ICT Projectsand Initiatives (Digital Dashboard Phase 1 report) we found that publicsector agencies’ financial and management processes did not enable thecomprehensive reporting of actual ICT expenditure across the public sector.During that audit, many agencies found it difficult to provide basic informationon their ICT spend and projects. Because of this difficulty, we had to estimatethe Victorian Government’s total ICT expenditure, which we conservativelyestimated to be about $3 billion per year.Within this total spend we found that, for capital ICT expenditure, agenciesspent an average of $720 million each year, from 2011–12 to 2013–14. Capitalexpenditure is usually the largest component of an ICT project’s budget so it is auseful indicator of total project expenditure. Typically, this type of expenditureis recorded as an asset in the balance sheet.We also found that agencies involved in the 2015 audit were not able to assureParliament and the Victorian community that their ICT investments had resultedin sufficient public value to justify the significant expenditure of taxpayers’money.As a result, we recommended that the Department of Premier and Cabinet(DPC) publicly report on ICT projects across the public sector. This reportingwas to include relevant project status information, such as costs, time lines,governance and benefits realisation.8 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportThe Victorian Government ICT DashboardDPC launched the Victorian Government ICT Dashboard (ICT Dashboard) inMarch 2016 to improve transparency and provide assurance that agencieshave efficiently, effectively and economically used public resources in theirICT projects.Agencies subject to the Financial Management Act 1994 (FMA) must providequarterly updates on their ICT projects valued at more than $1 million.Since the dashboard’s inception, 84 of the 184 agencies that are required toreport projects to the ICT Dashboard have reported on 439 projects. Therehave been 191 projects reported as complete, with a combined value of$907.9 million.In March 2018, DPC went live with a new dashboard system, which offers morefunctionality than the previous tool.The implementation of the ICT Dashboard has improved the transparency ofpublic sector ICT projects. Information reported on the ICT Dashboard isaccessible, interactive and easy to understand.It is also reasonably timely, taking into consideration the data handling andsign‐off processes in place for entering, checking and publishing the dashboarddata.It is reasonable for the public to expect information sources like the ICTDashboard to be authoritative and reliable. If they are not, then publicconfidence in the integrity of this information may be eroded.The information on the ICT Dashboard is ‘complete’ in terms of reportingagencies entering relevant and coherent data in mandatory fields for theprojects that they have disclosed.However, we are not able to give assurance on the overall completeness,accuracy or integrity of the data on the dashboard because: we detected a number of data errors for the projects we reviewed we detected some projects that were not reported by the agencies wereviewed we observed that nearly one‐third of all projects reported on the ICTDashboard were disclosed later than they should have been.Based on the anomalies we detected in a small subset of all the reported data,we suspect that this problem is more widespread.These inaccuracies show that DPC and agency processes are not adequate toproperly assure the integrity and reliability of source data, which is fundamentalto the overall accuracy and completeness of the ICT Dashboard.ConclusionVictorian Auditor‐General’s Report The Victorian Government ICT Dashboard 9Despite these accuracy and completeness challenges, the ICT Dashboard is amarked improvement in the availability and visibility of ICT project data. Thereare further opportunities for the ICT Dashboard to improve transparency as itmatures, particularly by: providing more useful and descriptive narratives on the purpose and statusof projects capturing and reporting expected project benefits better identifying and confirming what ICT category a project fits into.ICT Dashboard accessibility and ease of understandingThe ICT Dashboard is publicly available and can be viewed from any internetconnection. It is designed to be compatible with multiple devices such asdesktop computers, tablets or mobile phones.DPC conducted accessibility testing on the new ICT Dashboard before it waspublicly rolled out in March 2018. At the time of this audit, DPC stated that itintends to conduct regular accessibility audits of the ICT Dashboard.All the source data files used to populate the ICT Dashboard can be accessedand downloaded at the Victorian Government’s public data repository knownas data.vic.We downloaded this data into a typical spreadsheet program and observed thatit correctly tabulated against column headings and was consistent from quarterto quarter.ICT Dashboard timeliness and completenessBy its nature, the ICT Dashboard is a repository of data from public sectoragencies sent to DPC. This means that its content relies on the completeness,accuracy and candour of the data and status updates that agencies provide.Historically, DPC and the Department of Treasury and Finance (DTF) have takenan ‘arm’s length’ approach to their roles and responsibilities in assuring theaccuracy and integrity of government frameworks and reporting processes.DPC and DTF both stated in their response to the Digital Dashboard Phase 1report that they believed that primary responsibility for compliance withgovernment policies, including expenditure policies (such as specific ICTreporting requirements), rests with individual entities in accordance withVictoria’s financial management framework.If this approach to governance and oversight is to work effectively, thenreporting agencies need to be committed to, and capable of, providing accurateand complete data in a timely fashion.Findings10 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportTimelinessDue to the various processes required to collect, upload, review, approve andreport the ICT Dashboard data, there is a lag of at least three months beforedata is published.Many of these processes are discretionary, and DPC and reporting agenciescould streamline them to reduce the time taken from data collection to datapublication.However, process reform may not improve the frequency of dashboard updates.We observed that most of the agencies in this audit have limited capability toprovide updates in real time or more frequently than quarterly, due to internalresource and system constraints.This data lag means that the ICT Dashboard has limited utility as a managementsupport tool. For many fast‐moving ICT projects, a three‐month data delay canmean that what is publicly reported does not reflect what is actually currentlyhappening with the project.On a positive note, the upgraded ICT Dashboard, launched in March 2018, hasautomated some of the more laborious data entry and formatting activities thatDPC previously needed to do. This should free up DPC resources to give morefocus to higher‐order analytical tasks rather than mundane data validation anddata cleansing activities.CompletenessFor this audit, we considered two perspectives of completeness when reviewingthe dashboard.The first was about the completeness or integrity of the data reported to DPC,requiring all mandatory fields to be filled in with relevant data. We found noomissions in the uploaded data files that we reviewed. The new tool has aninbuilt data verification process that will not allow an agency to upload its dataunless all required fields are filled in with data that meets the requiredparameters.The second view of completeness relates to whether all the projects that shouldbe reported on the ICT Dashboard have been reported.During the audit, we identified five eligible ICT projects which the agenciesinvolved in this audit had not reported on the ICT Dashboard.Agencies are not consistently identifying whether their ICT projects will have, oralready have, reached the threshold of $1 million.Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 11ICT Dashboard reporting systems and processesThe agencies that we audited are very reliant on manual processes to identifyreportable projects. DPC and DTF have no real‐time visibility of agency financialsystems to help identify projects that should be reported on, apart from publicinformation such as media mentions, press releases or Budget Papers.In Victoria, the devolved financial accountability system means thatresponsibility for the accuracy and completeness of data rests with agencies.However, DPC does not have a process to assure government or the public thatagencies have correctly identified and reported all the ICT projects that shouldbe reported on the ICT Dashboard.As a consequence, there is limited oversight and assurance of the completenessand accuracy of reported projects and data. Errors in the reported data highlightthe need for both agencies and DPC to have stronger systems in place to reportaccurately.We tested 18 projects and their source documents in detail and found that theaccuracy of the information reported on the ICT Dashboard varies by agency—DPC, Melbourne Water (MW) and Public Transport Victoria (PTV) were mostlyaccurate, but the Department of Health and Human Services (DHHS) was notaccurate.Agencies’ compliance in disclosing and reporting their projects to theICT Dashboard in a timely manner also varied. Since the ICT Dashboardcommenced, 128 projects out of 439 (29 per cent) were reported laterthan they should have been. Agencies advised us that this can be due to thecost being unknown at the start of the project, or scope changes that increasethe cost of the project.Agencies have processes in place to report data to the ICT Dashboard, but theseprocesses do not always ensure that data is reported in accordance with the ICTReporting Standard for the Victorian Public Service (ICT Reporting Standard). Wefound instances where the project’s red/amber/green (RAG) status and itemsincluded in project budgets were inconsistent with the ICT Reporting Standard.For example, we found that the four projects we examined at MW and the oneproject we examined at DPC reported the whole project cost as delivery costs,rather than separately identifying the initiation and delivery costs, as requiredby the ICT Reporting Standard.Agencies have adequate processes to complete their mandatory ICT expenditurereporting, however, better coordination between Chief Financial Officers (CFO)and Chief Information Officers (CIO), or their equivalent, could help systematicallyidentify projects that need to be reported on the ICT Dashboard.12 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportWe recommend that the Department of Premier and Cabinet:1. amend the ICT Reporting Standard for the Victorian Public Service to: require that agencies provide a more descriptive and standardisednarrative about their ICT projects, including: information on the purpose of the project and overall valueproposition (see Section 2.5) a description of the expected impact on the efficiency andeffectiveness of service delivery (see Section 2.5) information on the benefits expected from the project’simplementation (see Section 2.5) require the capture and reporting of expected project benefits on theVictorian Government ICT Dashboard, including a capability forreporting agencies to monitor benefits realisation (see Section 2.5) clarify that any agency‐derived red/amber/green statuses used for aquarterly data update must align with the high‐level red/amber/greendefinitions specified by the Department of Premier and Cabinet toensure a consistent view across the public sector of ICT project status(see Section 3.4) require that the Chief Information Officer and Chief Financial Officer(or equivalent roles) jointly sign‐off the list of ICT projects thatunderpins the Financial Reporting Direction 22H reporting processand attest that all required projects have been identified and correctlyreported (see Section 3.2)2. continue to consult with agencies subject to the Financial ManagementAct 1994 to determine the most useful data fields to be included in theICT Reporting Standard for the Victorian Public Service with a key focus onavoiding any unnecessary reporting burden for agencies (see Section 2.4)3. conduct strategic analysis of ICT project categories and spend to supportthe intent of the Information Technology Strategy: Victorian Government2016–2020 for agencies to share existing solutions within the public serviceor identify services that could be transitioned into a shared services model(see Section 2.5)4. identify methods to review and confirm the accuracy and completeness ofdata reported on the Victorian Government ICT Dashboard andcommunicate the results back to agencies (see Section 2.2).We recommend that the Department of Treasury and Finance:5. implement a common chart of accounts across agencies subject to theFinancial Management Act 1994, to consistently capture and codeICT‐related expenditure, to allow better assessment and analysis acrossall these entities, regardless of their size or portfolio (see Section 3.2).RecommendationsVictorian Auditor‐General’s Report The Victorian Government ICT Dashboard 13We recommend that the Department of Health and Human Services, theDepartment of Premier and Cabinet, Melbourne Water and Public TransportVictoria:6. improve records management practices for ICT projects, giving particularattention to capturing and recording key project documents which showevidence of decisions and approvals (see Section 3.3).We have consulted with DHHS, DPC, DTF, MW and PTV. We considered theirviews when reaching our audit conclusions. As required by section 16(3) of theAudit Act 1994, we gave a draft copy of this report to those agencies and askedfor their submissions or comments.The following is a summary of those responses. The full responses are includedin Appendix A.DPC fully accepted four of the five recommendations directed to it, andaccepted the majority of the other recommendation. DPC stated that it sharesVAGO’s focus on improving transparency in ICT investments across governmentand provided an action plan for the recommendations it had accepted.DTF noted the findings of the report and accepted in principle therecommendation to establish a common chart of accounts to consistentlycapture and code ICT‐related expenditure. DTF agreed that improving thequality of performance data, both financial and non‐financial, is essential tobetter inform government policy decisions and prioritisation of resourceallocation, including for ICT investment.DHHS, DPC, MW and PTV accepted the recommendation directed to themto improve records management practices for ICT projects. DHHS committedto developing and communicating guidelines for the storing of key projectdocumentation and project status reporting. All agencies provided an actionplan detailing how they will address the recommendation.Responses torecommendations
Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 15Audit contextICT is integral to how governments manage information and deliver programsand services.Our April 2015 Digital Dashboard Phase 1 report found that, in general,government agencies’ financial and management processes did not enable thecomprehensive reporting of actual ICT expenditure across the public sector.During that audit, many agencies found it difficult to provide basic informationon their ICT spend and projects.We also found that agencies involved in the 2015 audit were not able to assureParliament and the Victorian community that their ICT investments had resultedin sufficient public value to justify the significant expenditure of taxpayers’money.As a result, we recommended that DPC publicly report on ICT projects acrossthe public sector. This reporting was to include relevant project statusinformation, such as costs, time lines, governance and benefits realisation.DPC, in conjunction with DTF, accepted this and all the other recommendationsdirected to it, and committed to implementing a quarterly report covering allICT projects with a budget greater than $1 million, by 31 March 2016.DPC noted that it did not have the authority to require departments andagencies to report the status of their ICT project, however, it would work withpublic sector bodies to request this information.DTF and DPC also noted that primary responsibility for compliance withgovernment policies, including with expenditure policies, rests with individualentities in accordance with the financial management framework.Victorian Government ICT expenditureThe Digital Dashboard Phase 1 report estimated that the VictorianGovernment’s total ICT expenditure was about $3 billion per year. This wasa conservative estimate because we found that financial processes in Victorianagencies did not enable comprehensive accounting of actual ICT expenditure.We found that agencies spent an average of $720 million on capital ICTexpenditure each year, between 2011–12 and 2013–14.1.1 Background16 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportFrom March 2016 to December 2017, agencies have reported 191 projects ascomplete, valued at $907.9 million. These are shown in Figure 1A.Figure 1ANumber of completed projects and reported cost, by quarter0204060801001201401601805 010152025303540
Q1 Q2 Q3 Q42016–17
CostProjects $ (million)Projects CostSource: VAGO, based on the data files available from data.vic.Information Technology Strategy: Victorian Government2016–2020The Information Technology Strategy: Victorian Government 2016–2020(the Victorian IT Strategy) was released in May 2016. The Victorian IT Strategygives direction and targets for public sector information management andtechnology over a five‐year time frame.The Victorian IT Strategy has four priorities:1. reform how government manages its information and data and makesthese transparent2. seize opportunities from the digital revolution3. reform government’s underlying technology4. lift the capability of government employees to implement ICT solutions thatare innovative, contemporary and beneficial.Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 17In response to one of our recommendations from the Digital Dashboard Phase 1report, one of the Victorian IT Strategy’s actions was to launch a publicdashboard covering ICT projects worth over $1 million.Other ICT dashboards across the worldGovernments in other countries and Australian states have establishedmechanisms to publicly report on their ICT projects.The format of this reporting varies, with some dashboards showing data at anaggregate level, such as expenditure by department, while others show verydetailed project‐specific information and even identify key individuals by nameand photograph.International ICT dashboardsUnited States of AmericaThe United States of America’s (USA) IT Dashboard was launched in 2009 toprovide information about 26 federal agencies’ IT expenditure and the progressof their projects.In March 2018, this dashboard contained data on over 7 000 projects, including700 identified as ‘major’ projects. Major projects are required to submit abusiness case detailing the cost, schedule and some performance data,however, there is not a dollar figure specifying when a project requires abusiness case.The USA IT Dashboard gives an overview of historical IT spending and forecastsfuture spending. In the 2019 fiscal year (1 October to 30 September), the USAexpects to spend $83.4 billion on ICT projects, as shown in Figure 1B. The USA ITDashboard also provides an overview of all projects’ risk ratings, as shown inFigure 1C.Figure 1BUSA’s total IT spending by fiscal yearSource: USA IT Dashboard, www.itdashboard.gov.18 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportFigure 1CUSA’s collated risk ratings for all reported ICT projectsSource: USA IT Dashboard, www.itdashboard.gov.The NetherlandsThe Netherlands’ dashboard, called the Central ICT Dashboard, providesinformation about major ICT projects, defined as valued at more than€5 million. In March 2018, this dashboard reported on 199 projects, with96 of them in progress.The dashboard compares total spend on multi‐year projects by ministries(equivalent to departments). The most recent data on the dashboard is currentto 31 December 2016.Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 19Australian ICT dashboardsQueenslandQueensland’s ICT Dashboard was launched in August 2013 and givesinformation on all major ICT projects overseen by the Queensland Government.By March 2018, the dashboard reported on 156 projects worth a total of$1.4 billion. Each department decides which projects to report on thedashboard.Queensland ICT projects use a RAG status rating. The dashboard providessummary statistics on projects by department, RAG status and expenditure perdepartment.Figure 1D shows the Queensland ICT Dashboard landing page, and summarysnapshot.Figure 1DLanding page of Queensland’s ICT DashboardSource: Queensland ICT Dashboard, www.qld.gov.au/ictdashboard.New South WalesNew South Wales launched the digital.nsw dashboard in November 2017. Thisdashboard reports on the New South Wales Government’s progress on projectsthat contribute to achieving its three digital government priorities.The website gives an overview of each project and its budget. Figure 1E displaysthe landing page and summary information.20 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportFigure 1ESummary information provided on digital.nswSource: New South Wales dashboard, www.digital.nsw.gov.au.Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 21DPC manages the Victorian Government ICT Dashboard, which was launched inMarch 2016. The ICT Dashboard is a reporting tool that shows key metrics frompublic sector ICT projects valued at more than $1 million.The ICT Dashboard fulfils a component of action 22 of the Victorian IT Strategy.The ICT Dashboard is available to anyone with an internet connection, isinteractive and allows users to see and filter data in different ways.Figure 1F shows the dashboard’s overview of ICT projects, as at April 2018.Figure 1FOverview of reported ICT projects on the ICT Dashboard, April 2018Source: ICT Dashboard.1.2 The VictorianGovernment ICTDashboard22 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportProject information is required for every ICT project that has a total projected,estimated, current or actual ICT cost of $1 million or more and is initiated or indelivery, or was completed, postponed or terminated, in the quarter.Since the dashboard’s creation, 84 of the 184 agencies that are required toreport on the ICT Dashboard have reported on 439 projects. The number ofreported projects has gradually risen since March 2016, as shown in Figure 1G.Figure 1GNumber of projects reported on the ICT Dashboard, by quarterNote: Until Q2 2017–18, agencies were required to report on projects until the end of the financialyear, even if the project was concluded. It was therefore expected that the number of projectswould increase throughout the financial year. From Q2 2017–18, DPC advised that it would nowremove projects from the dashboard as they were completed, terminated or merged.Source: VAGO, based on the data files available from data.vic.ICT projects reported on the ICT DashboardThe latest available data, for the December quarter 2017, shows 246 projectsfrom 65 agencies. These projects are valued at $1.8 billion.Figure 1H shows the number of projects reported and the budgeted cost, bygovernment domain. The Health domain has the largest number of projects at55 and second highest cost at $337.1 million. The Law and Justice domain hasthe highest combined project cost of $730.2 million.
Q1 Q2 Q3 Q42016–17
229050100150200250300350ProjectsVictorian Auditor‐General’s Report The Victorian Government ICT Dashboard 23Figure 1HNumber and cost of projects, by government domain, December quarter 2017Source: VAGO, based on the data files available from data.vic.Agencies are required to report on their projects’ current implementation stage.The dashboard has seven implementation status categories, as shown inFigure 1I.Figure 1IStages of implementation
Pre ‘project delivery’ activities are underway includingpreliminary planning, feasibility study, business casedevelopment and/or funding request.
Business CaseApprovedProject business case has been approved or funding has beenallocated.Delivery Project has commenced delivery.
Project has delivered its outcomes, and is being, or has beenclosed.Project has been temporarily put on hold.Terminated prior to‘project completed’Work on the project has ended prematurely (i.e., a decisionhas been made to stop work).Merged Scope of the project has been merged into another project.Source: Version 2.0 of the ICT Reporting Standard.010020030040050060070080001020304050607080Projects CostCost($ million)Projects24 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportUnder version 1.0 of the ICT Reporting Standard, agencies reported againstsix stages: initiated in progress—pre‐implementation activities in progress—implementation activities project completed—project closed project postponed project terminated (prior to completion).As shown in Figure 1I, version 2.0 of the standard has not significantly alteredthe project stages.Figure 1J shows the breakdown of the 246 reported projects, by stage. Themajority (149, or 61 per cent) are listed as in ‘delivery’.Figure 1JProjects by stage of implementation, December quarter 2017Source: VAGO, based on the data files available from data.vic.The ICT Dashboard reports the status of projects using a RAG format. Agenciesreport the status of the project, as per their most recent project control board(or equivalent) report, using the definitions outlined in Figure 1K.Under version 1.0 of the ICT Reporting Standard, the RAG status used thesame definitions, however, the status only reflected whether the currentimplementation time line was on schedule. It did not take account of othermatters such as risk, functionality or cost.
1490 20 40 60 80 100 120 140 160Closed—terminatedClosed—mergedClosed—completedPostponedDeliveryBusiness Case ApprovedInitiatedProjectsVictorian Auditor‐General’s Report The Victorian Government ICT Dashboard 25Figure 1KRAG status definitionsStatus DefinitionOn trackIssues exist but they are under the project manager’s controlSerious issues exist that are beyond the project manager’s controlSource: Version 2.0 of the ICT Reporting Standard.There is no guidance in the ICT Reporting Standard regarding when a projectneeds to start reporting a RAG status. DPC advised that its new online reportingtool has built‐in functionality to activate RAG status reporting when a projectmoves into the delivery stage. However, this has not been observed or tested bythe audit team.In the December quarter 2017, 178 of the 246 projects (72 per cent) reporteda RAG status.Figure 1L shows the RAG status of reported projects, for the December quarter2017. Just under 80 per cent of projects were reported as on track.Figure 1LReported projects by RAG status, December quarter 2017Source: VAGO, based on the data files available from data.vic.142351GreenAmberRed26 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportNew dashboard tool launched in early 2018In March 2018, DPC went live with a new dashboard system, which offers morefunctionality than the previous tool.The new tool continues most of the previous dashboard’s public‐facingfunctionality and is highly interactive. It allows a user to receive a high‐leveloverview, such as RAG status, domain or ICT project type, or to specifically focuson a single project or agency. Users can apply filters for project costs andduration.The new dashboard home page has three categories for users—understand,explore and improve.The ‘understand’ section provides the dashboard’s strategic objectives and datacollection process, and discusses how the data is used.New features give users a historical overview of all ICT projects. Users candownload quarterly data files and, as of March 2018, this download includeshow projects have progressed over the previous five quarters.The ‘explore’ section allows users to explore the reported government ICTprojects across stage, status and domain. A dashboard overview summarisesprojects by stage and status, by cost tiers, top five departments/agenciesranking, government domains and ICT project categories.A search, filter and sort function has been added in the new dashboard systemto improve useability. A revision history is now available for each agency to trackany changes to project schedules and costs.The new dashboard system can also generate a ‘project on a page’ summary,which gives agencies snapshots of their projects’ status and data.The last section is ‘improve’ where users can provide feedback to dashboardadministrators.Financial Management Act 1994The FMA governs Victorian Government financial investments and reporting.The FMA requires government to apply the principles of sound financialmanagement.In the context of this legislation, sound financial management is to be achievedby: establishing and maintaining a budgeting and reporting framework prudent management of financial risks, including risks arising from themanagement of assets considering the financial impact of decisions or actions on futuregenerations disclosing government and agency financial decisions in a full, accurate andtimely way.1.3 Legislationand standardsVictorian Auditor‐General’s Report The Victorian Government ICT Dashboard 27Under section 8 of the FMA, the Minister for Finance issues directions, whichapply to public sector bodies.These include Financial Reporting Directions (FRD), which are mandatory andmust be applied if the public agency is subject to the FMA.Financial Reporting Direction 22H—Standard disclosuresin the report of operationsThe purpose of FRD 22H is to prescribe the content of agencies’ report ofoperations within their annual report to ensure consistency in reporting.One of the requirements of FRD 22H is for agencies to disclose total ICTbusiness as usual (BAU) expenditure and total ICT non‐BAU expenditure forthe relevant reporting period.This part of FRD 22H was first applied in the 2016–17 financial year andattestations on ICT expenditure were included in agencies’ annual reports forthat period.ICT Reporting Standard for the Victorian Public ServicePrior to the issuing of the FRD 22H reporting requirements for ICT expenditure,DPC released version 1.0 of the ICT Reporting Standard in September 2015.The ICT Reporting Standard sets the business reporting requirements for ICTexpenditure and ICT projects with budgets over $1 million and applies to alldepartments and bodies, as defined by the FMA.At the same time, DPC released the ICT Expenditure Reporting Guideline for theVictorian Government. The guideline gives data reporting recommendations andguidance to agencies.In September 2017, DPC issued version 2.0 of the ICT Reporting Standard.Version 2.0 references the requirement for agencies to comply with FRD 22Hand added seven extra reporting fields.The revised ICT Reporting Standard was first used for the 2017–18 Decemberquarter data which was released to the public in late March 2018.There are a number of governance bodies that are relevant for ICT: The Minister for Finance is the responsible minister under the FMA and alsoissues FRDs. The Special Minister of State is responsible for the Victorian IT Strategy,including the ICT Dashboard. The Victorian Secretaries’ Board (VSB) comprises the Secretaries of eachdepartment, the Chief Commissioner of Victoria Police and the VictorianPublic Sector Commissioner. The aim of the VSB is to coordinate policyinitiatives across the public sector, and promote leadership and informationexchange. The VSB was the approval authority for the original and revisedICT reporting standards.1.4 Governanceframework28 The Victorian Government ICT Dashboard Victorian Auditor‐General’s Report The Integrity and Corporate Reform subcommittee of the VSB wasdelegated responsibility, by the VSB, for dealing with operational matters inrelation to the ICT Dashboard. The Victorian CIOs’ Leadership Group (formerly known as the CIOs’ Council)includes the CIOs from bodies who are members of the VSB plus any invitedobservers. The group is responsible for setting technical ICT standards andguidelines and for coordination of major ICT issues across agencies.The following describes the departments with responsibilities for the ICTDashboard and gives information on the agencies that we included in this auditfor focused testing.Department of Premier and CabinetDPC is the owner of the ICT Dashboard and oversees the reporting process. Itcollects data from FMA agencies and publishes quarterly reports on the ICTDashboard.DPC is also a reporting agency and has reported six projects on the ICTDashboard. We reviewed one of DPC’s ICT projects in detail.Department of Treasury and FinanceDTF is the owner of the overall State Budget process that is key to the fundingapproval for most ICT projects. DTF also manages the High Value High Risk(HVHR) process which requires the ongoing monitoring of more complex,higher‐risk ICT projects.As an entity, DTF has reported three projects on the ICT Dashboard since itlaunched in March 2016, with two of these currently in the initiation stage.Department of Health and Human ServicesDHHS develops and delivers policies, programs and services that support andenhance the health and wellbeing of all Victorians.DHHS has reported 51 projects on the ICT Dashboard since the dashboard’sinception. We reviewed eight of DHHS’s ICT projects in detail.Melbourne WaterMW is a statutory authority that manages and protects Melbourne’s majorwater resources on behalf of the community.MW has reported 33 projects on the ICT Dashboard since its launch. Wereviewed four of MW’s ICT projects in detail.1.5 Agency rolesVictorian Auditor‐General’s Report The Victorian Government ICT Dashboard 29Public Transport VictoriaPTV is a statutory authority that acts as a system coordinator for all publictransport in Victoria. It aims to improve public transport in Victoria, by ensuringbetter coordination between modes, facilitating expansion of the network,auditing public transport assets and promoting public transport.PTV has reported 20 projects on the ICT Dashboard since its launch. Wereviewed five of PTV’s ICT projects in detail.Our Digital Dashboard Phase 1 report estimated that the VictorianGovernment’s ICT expenditure was $3 billion per year and found thatVictorian agencies did not have comprehensive accounting in place toidentify actual ICT expenditure.Our follow‐up March 2016 report Digital Dashboard: Status Review of ICTProjects and Initiatives – Phase 2 examined a selection of ICT projects in detail.Although some elements of better practice were identified, the audit confirmedthat the Victorian Government needed to be more effective in planning andmanaging ICT projects, as they continued to show poor planning andimplementation, resulting in significant delays and budget blowouts.ICT projects are a significant component of the Victorian public sector’s annualexpenditure.It is important to track information on the status and outcomes of public sectorICT initiatives to monitor whether public resources have been spent in anefficient and effective manner. Greater transparency also helps to assesswhether these investments have enhanced government services or addressedthe problems they were meant to resolve.The ICT Dashboard was developed to provide accurate, reliable and completeinformation about ICT expenditure by Victorian Government departments andagencies and to provide assurance and transparency to the public.The ICT Dashboard has now reported eight quarters of data and this audit givessome insight into whether agencies are now more reliably monitoring andrecording—as well as transparently reporting—their ICT investments.The objective of this audit was to examine whether transparency in governmentICT investments has improved since the development of the ICT Dashboard.We examined whether: information reported on the ICT Dashboard is accurate, timely andcomplete information reported on the ICT Dashboard is accessible and easy tounderstand agencies have systems and processes to ensure that data reported on theICT Dashboard meets the ICT Reporting Standard.1.6 Previousaudits1.7 Why this auditis important1.8 What thisaudit examinedand how30 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportWe examined DPC in its role as the system owner of the ICT Dashboard and alsoexamined one of its ICT projects which is also scrutinised by DTF’s HVHRprocess.We examined DTF due to its role in overseeing the State Budget process,financial compliance frameworks and the HVHR process.DHHS, PTV and MW were included in the audit to allow us to examine a spreadof projects across different areas of the public sector. Across these three entitieswe examined 17 ICT projects in detail.These projects were chosen on a risk and materiality basis, which includedfactors such as schedule, cost, type of project and reported RAG status.The full list of projects we examined in detail can be found in Appendix B.The methods for this audit included: a review of corporate documents, including project plans, policies,frameworks, and briefings at selected departments and agencies interviews with staff and senior management at selected departments andagencies a review of processes and key information systems used to collect, analyseand report information to the ICT Dashboard a review of financial data related to the case study ICT projects andexpenditure at selected departments and agencies.We conducted our audit in accordance with section 15 of the Audit Act 1994and ASAE 3500 Performance Engagements. We complied with theindependence and other relevant ethical requirements related to assuranceengagements. The cost of this audit was $310 000.The structure of this report is as follows: Part 2 discusses our review of the ICT Dashboard Part 3 discusses our review of case study projects at selected agencies.1.9 ReportstructureVictorian Auditor‐General’s Report The Victorian Government ICT Dashboard 31The ICT Dashboard—transparency andoversightThe ICT Dashboard launched in March 2016 and, since then, there have beeneight quarters of data reported. From the potential 184 FMA agencies requiredto report on the ICT Dashboard, 84 have reported 439 ICT projects.The ICT Reporting Standard requires applicable agencies to report data on theICT Dashboard for projects worth over $1 million. The ICT Reporting Standard’skey objectives are to: increase government transparency in managing ICT expenditure and projectstatus identify emerging trends in ICT expenditure and project types discover collaboration and shared services opportunities promote consistency in tracking ICT expenditure and the performance ofICT projects enable clear oversight for a more effective approach to future informationmanagement and technology meet our Digital Dashboard Phase 1 report audit recommendations.This part of the report discusses our review of the application of the ICTReporting Standard, with a particular focus on the transparency, accuracy andusefulness of the data that the ICT Dashboard displays.32 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportThe information reported on the ICT Dashboard is accessible and easy tounderstand. It is also reasonably timely, taking into consideration the processesthat need to be followed for the data to be collected and published.The information is also ‘complete’ from a data entry perspective for the projectsthat agencies are reporting. However, we are not able to give assurance that allprojects that should be included in the dashboard were reported, because wedetected a number that had been omitted. We found the omissions were due tohuman error and had no discernible pattern.Although the ICT Dashboard has been a marked improvement on the previousquality and availability of ICT project data, it could further mature and improvetransparency, particularly by: providing more useful descriptive narratives of the purpose and status ofprojects capturing and reporting expected project benefits better identifying and confirming what ICT category a project fits into, suchas ‘records management’.Since its launch, the ICT Dashboard has created a substantial increase in thequality and availability of information about public sector ICT projects.Our Digital Dashboard Phase 1 report identified the need for such a tool due tohistorically poor transparency within the public sector around what was beingspent on ICT, in which agency and for what purpose.In response to our Digital Dashboard Phase 1 report, both DPC and DTF agreedthat there was a need to set up an ICT‐specific dashboard.The Victorian IT Strategy authorised the establishment of the ICT Dashboard,and the ICT Reporting Standard gives operational authority to the content andrequirements of the ICT Dashboard. The ICT Reporting Standard was approvedby the VSB in September 2015 and updated in September 2017.The ICT Reporting Standard also notes that where the project includesnon‐ICT‐related investment, agencies should only report on the ICT projectcomponent.By its nature, the ICT Dashboard is a repository of data from public sectoragencies sent to DPC. This means that its content is driven by the completeness,accuracy and candour of the data that agencies provide when responding to themandatory elements of the ICT Reporting Standard.An apparent transparency challenge is that reporting agencies are notconsistently identifying whether their ICT projects will have, or already have,reached the ICT Reporting Standard’s threshold of $1 million.The agencies that we audited rely on staff manually checking lists extractedfrom different systems to identify reportable projects. DPC and DTF have noreal‐time visibility of agency financial systems to help identify projects thatshould be reported, apart from public information sources such as mediamentions, press releases or Budget Papers.2.1 Conclusion2.2 Has thedashboardimprovedtransparency?Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 33During the audit, we identified five eligible ICT projects that agencies did notreport on the ICT Dashboard. We discuss this in detail in Part 3.DPC does not have a process to assure itself, government or the public thatagencies have correctly identified and reported all ICT projects that should beon the ICT Dashboard. During this audit DPC stated that it does not believe ithas a role in assuring the data that agencies provide.The devolved financial accountability system in Victoria means that theresponsibility for the accuracy and completeness of data rests with agencies.The consequence of this approach is that there is limited oversight of andassurance about the accuracy and completeness of reported data.Likewise, the agencies where we detected errors and omissions do not haveadequate processes to make sure the data they report is accurate and complete.The ICT Dashboard is publicly available and can be viewed from any internetconnection. It is designed to be compatible with multiple devices such asdesktop computers, tablets and mobile phones.Accessibility of the ICT Dashboard toolIn addition to the continuous availability of the ICT Dashboard for anyinternet user, DPC states that it strives for the ICT Dashboard to complywith the World Wide Web Consortium’s Web Content Accessibility Guidelines(WCAG) 2.0. These guidelines are designed to help people with disabilitieswho face challenges when accessing material on the internet.The Victorian Government has publicly stated that it will ensure its onlinecontent is available to the widest possible audience, including readers usingassistive technology or other accessibility features.DPC conducted accessibility testing of the new ICT Dashboard tool beforeit publicly rolled out in March 2018. DPC also advised us that it hadcommissioned a disability advocacy agency to conduct a user experience testfor people with impaired vision. At the time of this audit, DPC stated that itintends to conduct regular accessibility audits of the ICT Dashboard.Accessibility of the ICT Dashboard dataDuring the audit, we were able to access and download all the source data filesthat had been used to populate the ICT Dashboard since it was launched.A data file for the most recent quarter is available on the dashboard’s‘understand’ page, while previous quarters are located on the VictorianGovernment’s public data repository known as data.vic.2.3 Is it easy tounderstand andaccessible?34 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportThere are no hyperlinks to historical datasets from the ICT Dashboard page todata.vic and identifying and downloading historical files was not a simpleprocess.We downloaded the data into a typical spreadsheet program and observed thatit correctly tabulated against column headings and had consistency fromquarter to quarter.We identified some minor discrepancies related to project codes and namingconventions. We understand DPC resolved this when transitioning to the newICT Dashboard.The technology used to display the ICT Dashboard data has matured over thelast two years.The new ICT Dashboard is a cloud‐based ICT platform which allows reportingagencies to directly upload required data into the system. The most recent dataupload (December quarter 2017) is now in the cloud‐based database anddisplayed through a data visualisation tool.This is an efficiency improvement over the previous system, which requiredagencies’ spreadsheet data files to be uploaded into a shared repository forchecking and formatting offline by DPC, before being imported into a datavisualisation tool for online publication.The new system’s data entry approach has streamlined the review process foragencies and DPC because validation checks are now built into data fields thatwill only accept a data input that meets required parameters.Once entered into the system, the data is staged from ‘draft’ to ‘approved’ bythe reporting agency, then held by DPC as ‘confirmed’ until all the dashboarddata is ‘published’ at the end of the quarter, after the Special Minister of Stateis provided with a briefing.Is the dashboard timely?Due to the various processes required to collect, upload, review, approve andreport the ICT Dashboard data, there is a lag of at least three months beforedata is published on the dashboard.This means that data could be nearly six months old just before the dashboardhas its quarterly refresh.For many fast‐moving ICT projects, this can mean that what is publicly reporteddoes not reflect what is actually currently happening.2.4 Is theinformation timelyand complete?Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 35We observed that, at present, there are limited opportunities to reduce thisdata lag due to: manual and time‐consuming data collation and data entry processes atagencies agency internal sign‐off for RAG status and progress commentary cross‐checking with DTF information related to ICT projects that are subjectto HVHR processes DPC‐initiated reviews of any major data anomalies and projects withconsecutive red or amber RAG statuses a quarterly briefing to the Special Minister of State, which gives a summaryof the ICT Dashboard’s results prior to publication.On a positive note, the new tool has automated some of the more laboriousdata entry and data formatting activities which should free up DPC resources togive more focus to higher order analytical tasks rather than mundane datavalidation and data cleansing activities.Is the dashboard complete?Between March 2016 and September 2017, agencies reported against 17 datafields. Since December 2017, when version 2.0 of the ICT Reporting Standardbecame operational, agencies have reported against 24 fields.A complete list of the data fields agencies need to report against can be found inAppendix C.For this audit, we considered two views of completeness when reviewing thedashboard.The first view was about the completeness or integrity of the data reported toDPC, requiring all mandatory fields to be filled in with relevant and coherentdata.We reviewed the data for the September 2017 quarter that was published inDecember 2017. We found no omissions in the mandatory data fields that wereviewed.The new tool has an in‐built data verification process that will not allow anagency to upload its data unless all required fields are complete. An incompleteupload is listed as ‘draft’ and will attract a follow up from DPC if it is notactioned promptly by the reporting agency.The second view of completeness related to whether all the projects that shouldbe reported on the ICT Dashboard have been reported.36 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportAs stated earlier, we identified five eligible ICT projects that agencies had notreported on the ICT Dashboard. We discuss this further in Part 3 of this report.We also found that a significant number of projects have not been reported onthe dashboard according to the time frame set by the ICT Reporting Standard.We analysed when a project commenced and when the project was firstreported on the ICT Dashboard. We found that 128 of 439 (29 per cent) projectswere reported later than they should have been. The longest disclosure delaywas 21 months late for two projects, followed by 18 months late for threeprojects. The average delay in disclosure was seven months.Between March 2016 and the end of December 2017, the ICT Dashboard sitehad about 10 400 unique page views from internet users. The tracking softwarefor the site does not show which domain these users come from so it is notpossible to attribute these views to a particular type of user.The new system’s functionality increases the opportunity for better oversight ofall ICT projects. In particular, showing RAG status trends over time will give usersof the ICT Dashboard a longer‐term view of how a project is progressing.The new system also allows reporting agencies to use non‐mandatory fields inthe system to track a wider range of project attributes beyond the cost orschedule data, such as issues, risks and benefits.Identifying opportunities to better align investmentsacross agenciesOne of the actions of the Victorian IT Strategy is ‘Following on from the ICTProjects Dashboard, establish a high level “portfolio management office”function to form a portfolio view of government ICT projects over $1 million,providing templates, advice and assistance where relevant’.Establishing this function requires data to be visible across all portfolios, to gaina broad view of ICT projects across the public sector. The current dashboard hassubstantively achieved this.However, to move to a more strategic view, the ICT Dashboard should aim toprovide more meaningful, qualitative information on the nature, purpose andbenefits of an ICT project. It also needs to categorise ICT projects consistently.Quality and usefulness of project descriptive narrativesOne of the goals of the ICT Reporting Standard is to identify emerging trends inICT expenditure and project types across government.The latest ICT Dashboard data has taken the first step towards collectingstandardised ICT project category descriptions. The 30 project type descriptionsare from the Australian Government Architecture Reference Models published in2011 by the now defunct Australian Government Information ManagementOffice.2.5 Does itprovide betteroversight?Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 37At present, it is difficult to derive a detailed understanding of the nature andpurpose of a reported ICT project from the descriptive material that agenciesreport on the ICT Dashboard.Project titles do not allow users to understand the functionality or capability ofthe project or why it is being implemented.We also observed that although the project type descriptor is standardised, DPChas included an extra category of ‘other’ and allows agencies to select multiplecategories for their ICT projects. DPC advised us that the ‘other’ option wasprovided because the project category field is mandatory.DPC also advised us that where an agency selected ‘other’, DPC worked with theagency to assist them to choose a more relevant field, and that currently noagencies have projects categorised as ‘other’ on the ICT Dashboard.We query whether allowing multiple selections or the ‘other’ category willprovide a more granular view of ICT investments, without some level ofjustification by the agency or more quality control and oversight by DPC.Benefits and impacts from ICT projectsOur Digital Dashboard Phase 1 report found that Victorian agencies and entitieswere not in a position to assure Parliament and the Victorian community thattheir ICT investments resulted in sufficient public value to justify the significantexpenditure of taxpayers’ money.This was because the agencies were unable to demonstrate that the expectedbenefits from ICT investments had been realised. At the time of the DigitalDashboard Phase 1 report, only a quarter of the 1 249 projects that werereported to VAGO had a benefits realisation plan. Only 33 per cent of thereviewed sample effectively laid out the expected benefits and set out measuresand targets for these.The Digital Dashboard Phase 1 report also found that it was very difficultto obtain consistent and meaningful data on benefits realisation. Of the788 projects reported as ‘completed’, a little over 10 per cent had had theirexpected benefits assessed.Based on these findings, we recommended establishing a public reporting (PR)mechanism that provides relevant project status information on ICT projectsacross the public sector, with key metrics and project information to be includedin this reporting such as costs, time lines, governance and benefits realisation.Low rates of tracking or assessment of benefits realisation is not confined to theICT domain. We have commented in other reports that a lack of a methodicalevaluation culture in the Victorian public sector often means there is nosystematic and objective collection of lessons learned, to better inform theplanning and execution of future projects.We have observed this deficiency in audits of projects across many sectors andhave also identified a comparatively low number of post‐implementationGate 6 ‘Benefits Realisation’ reviews done under DTF’s gateway review process,compared to other earlier stage gates.38 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportOur review of the new ICT Dashboard software tool identified that it hascapacity to include more information about ICT projects such as issues,stakeholders, risks and benefits.We understand that the system also has a purpose‐designed module whichagencies can use to track benefits for each of their reported ICT projects.To fully meet the benefits realisation component of our 2015 recommendation,DPC should promptly examine the implementation of this functionality. DPCshould also amend the ICT Reporting Standard to require agencies to report onthe expected impact of their ICT investments and to track benefits as they arerealised.Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 39The ICT Dashboard—agency data and reportingIn Victoria’s devolved financial accountability system, responsibility for theaccuracy and completeness of data rests with the board or accountable officerof each entity, and is certified through the entity’s governance process, ratherthan by DPC or DTF.This part of the report discusses the results of our testing of the data reportedon the ICT Dashboard by the agencies involved in this audit.The accuracy of the information reported on the ICT Dashboard varies byagency. We found that the information entered by DPC, MW and PTV wasmostly accurate, but the information entered by DHHS was not accurate.Agencies’ compliance in reporting projects to the ICT Dashboard was varied. Wefound two projects at MW were not disclosed for 18 months and one project atDHHS was not reported until 15 months after it commenced.Project‐related data reported on the ICT Dashboard is complete, in that allnecessary fields have been filled in correctly, however, because we detectedfive projects that were omitted in the agencies we reviewed, we cannot be surethat all ICT projects that should be reported have been reported.Agencies have manual processes to report data on the ICT Dashboard, butthese processes do not always ensure that data is reported in accordance withthe ICT Reporting Standard. We found instances where a project’s RAG statuswas inconsistent with the ICT Reporting Standard’s definition, as well asinconsistencies in recording items in project budgets, such as the allocationof staffing costs.Agencies have adequate processes to complete their mandatory reporting onICT expenditure, however, better coordination between CFOs and CIOs (or theirequivalent) could help systematically identify ICT projects that should bereported on the ICT Dashboard.3.1 Conclusion40 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportThe current ICT Reporting Standard defines 24 data fields that agencies arerequired to publish on the ICT Dashboard. It also specifies agency requirementsfor reporting their BAU and non‐BAU ICT expenditure in their annual financialreports.For this audit, we focused on the fields reported on the dashboard and non‐BAUexpenditure. We did not assess BAU expenditure, apart from examining themost recent attestation made by the agencies in this audit, as ICT projects arenot typically funded from this expenditure category.Are ICT Reporting Standard requirements being met?We expected to see that agencies were timely in identifying and reportingrelevant projects on the ICT Dashboard, in accordance with the ICT ReportingStandard.High‐level results by agencyThe four agencies we reviewed are meeting most of the ICT Dashboardreporting requirements. However, we observed some anomalies and smallerrors, which shows that they are not consistently meeting the ICT ReportingStandard.We found that some agencies were using different criteria to determine theirRAG status than those required by version 1.0 of the ICT Reporting Standard.There was also often a delay between the start of the project and when theproject was reported on the ICT Dashboard.We also detected examples where a project was omitted from the dashboard.We discuss this further in the section on mandatory ICT expenditure reportingrequirements.Department of Health and Human ServicesWe found that although DHHS generally follows the ICT Reporting Standard, itdid not follow the standard for all projects when determining RAG status.DHHS previously defined and reported its RAG status based on a combination ofschedule, budget and risks, whereas under the version 1.0 of the ICT ReportingStandard, the RAG status should only have been based on schedule.We found that half of the projects we reviewed were late in being reporting onthe dashboard: One project commenced in May 2016 and was reported inDecember 2016—seven months after starting. One project that commenced in June 2016 was not reported untilSeptember 2017—15 months after starting. One project commenced in December 2016 but was not reported untilSeptember 2017—nine months after starting. One project commenced in February 2017 and was first reported inSeptember 2017—seven months after starting.3.2 Are agenciesfollowing the ICTReportingStandard?Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 41DHHS advised us that during the early stages of analysis and planning a project,the scope of the project may increase or decrease. As a result, some projectsare not immediately identified as likely to exceed the $1 million threshold andhave to be reported on the ICT Dashboard.Department of Premier and CabinetWe assessed one project at DPC and found that it did not report an initiationcost, but reported all costs as implementation costs.The project team advised that the project’s use of two different projectmanagement methodologies meant that classifying the phases of the projectas initiation or delivery was difficult. In order to provide some transparencyabout the project’s cost, DPC decided to report the full amount asimplementation costs.DPC advised that some of the product concepts it developed during theplanning phase actually delivered some aspects of the final product. So, whilethese concepts included both initiation and delivery costs, the costs were onlyreported as delivery costs.Melbourne WaterMW adhered to the ICT Reporting Standard, except for reporting of initiationcosts and RAG status.MW does not separately report initiation costs, with all costs reported asdelivery costs. This means that there is no transparency about planning costs.MW advised that it believed that, at the time the report was developed, it wasnot practicable to reliably automate the reporting of separate initiation anddelivery costs.MW previously reported its RAG status based on the overall status of theproject, whereas under version 1.0 of the ICT Reporting Standard, the RAGstatus should have only been based on schedule.MW previously only reported projects to the ICT Dashboard when the projectwas confirmed as proceeding. This resulted in MW reporting 19 projects laterthan they should have been. MW told us that it sought advice from DPC in late2017 regarding when projects should start being reported on the ICT Dashboardand, as a result of that advice, will now report projects earlier in their planningphase.This earlier identification and reporting of projects will be a positive steptowards transparency.Public Transport VictoriaFor the projects that we assessed at PTV, we found that the agency adheres tothe ICT Reporting Standard.However, PTV may face challenges reporting its project RAG status goingforward, as the RAG status definitions were altered in version 2.0 of the ICTReporting Standard and PTV’s internal RAG status definitions do not match therevised definitions.42 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportThis means that if PTV were to report internally that a project had a ‘red’status, it would not necessarily equate to the ‘red’ status description in theICT Reporting Standard, that ‘serious issues exist and they are beyond theproject manager’s control’.PTV advised us that there is a variation between its internal RAG statusdefinitions and the ICT Reporting Standard definitions because PTV hasimplemented a project methodology that it believes suits the wide rangeand type of projects that it delivers.FRD 22H requirementsFRD 22H prescribes ICT expenditure disclosures that agencies must make intheir annual report of operations.Under the Standing Directions of the Minister for Finance, the accountableofficer or governing board of an agency must attest—that is, personally sign anddate—this disclosure within the annual report of operations.Attestation assurance processes in agenciesAll the agencies we examined completed the required FRD 22H attestation intheir 2016–17 annual report.Because the FRD 22H attestation is a component of the report and not acomponent of FMA agencies’ yearly financial statements audited by VAGO,there is no external assurance provided in regard to its accuracy.Non‐BAU ICT expenditure disclosureFRD 22H requires that entities report their total non‐BAU ICT expenditure,broken down between operational expenditure and capital expenditure.We examined the non‐BAU expenditure for agencies in this audit, to identify ifany ICT projects were underway that met the $1 million reporting threshold butwere not included on the ICT Dashboard.We also asked agencies to provide us with a list of their current ICT projects,including the name of the project, start and expected completion date andforecast or approved total cost. We used this list to cross‐check informationreported under FRD 22H and ICT projects reported on the ICT Dashboard.Department of Health and Human ServicesAll DHHS’s ICT projects that should have been reported were correctly disclosedon the ICT Dashboard, although four projects we reviewed were disclosed late.Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 43Department of Premier CabinetWe identified one project that should have been reported on the ICT Dashboardearlier than it was. DPC identified this project through its internal financialmanagement processes when project expenditure reached $1 million, however,the project’s projected cost had been known for at least six months prior to this.As DPC does not centrally track projects until project expenditure reaches$1 million, DPC cannot be sure that it is reporting all the projects that should beon the ICT Dashboard.Department of Treasury and FinanceAll DTF’s ICT projects that should have been reported were correctly disclosedon the ICT Dashboard.Melbourne WaterWe found one project currently underway at MW that should have beenreported on the ICT Dashboard but was not.After we told MW about this, it advised us that this project had been misseddue to a data entry error in its project management tool. The project’s finaldelivery cost was under $1 million but should have been reported under theprocess that MW follows. MW advised us that it has since changed to a newproject management tool where reporting is set up in a way that this particularerror can no longer occur.Public Transport VictoriaWe identified four projects that were not, but should have been, reported onthe ICT Dashboard.PTV advised us that two of these had the status ‘postponed’ when the ICTDashboard launched in March 2016, and were not reported after the projectsrecommenced. The third project was omitted due to an administrative error.The two projects that were postponed have since been completed and the thirdwill be included in the March 2018 quarter reporting, which is expected to bepublished on the ICT Dashboard in June 2018.The fourth missing project is a joined‐up project with the Department ofEconomic Development, Jobs, Transport and Resources (DEDJTR), which wasmissed due to an assumption by PTV that the project would be reported byDEDJTR.PTV told us that it did not think it needed to report this project, as it was part ofa broader DEDJTR project. However, DEDJTR only reports its component of thisproject on the ICT Dashboard, not the PTV component. PTV has rectified thisoversight, with the project included in the March 2018 reporting upload, whichis expected to be published in June 2018.44 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportFor the ICT Dashboard to effectively improve the transparency of ICT projects,it is important that the data underpinning agency reporting is verifiable andaccurate. We assessed the accuracy of the data reported on the ICT Dashboardby reviewing 18 projects from four agencies.To determine the accuracy of this data (dates, dollar figures, RAG status), weassessed whether the data on the dashboard was verifiable against sourcedocuments held at the agency.We found that the reported data is mostly accurate in three of the four agencieswe tested, however one agency had a larger number of inaccuracies, as shownin Figure 3A.Figure 3AAgencies reviewed and overall accuracy assessment for selected projectsAgencyNumber of projects reportedon the ICT Dashboard(1 March 2016 to30 September 2017)Number ofprojects weexaminedOverall accuracyassessmentagainst projectswe examined
Not accurateMostly accurateMostly accurateMostly accurateSource: VAGO and data from the data files, available from data.vic.We found that the reviewed agencies had the most difficulty when providingdocumentation to verify the initiation phase end date and cost of the initiationphase.A summary of results from our 18 detailed project data reviews is in Appendix B.High‐level results by agencyWe expected that agencies would be able to provide us with sourcedocumentation, such as project status reports, project initiation approvals andapproved budgets and time lines, for the 17 data fields that were reported onthe ICT Dashboard under version 1.0 of the ICT Reporting Standard. This versionwas applicable between March 2016 and September 2017.Appendix C contains details of the data fields required under both versions ofthe ICT Reporting Standard.3.3 Are agenciesreporting accuratedata?Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 45Department of Health and Human ServicesDHHS has reported 51 projects since the ICT Dashboard’s inception. For theDecember quarter 2017, DHHS is reporting 32 projects, which are valued at$132.6 million.Figure 3B displays the implementation stages of these projects, and their status.Figure 3BDHHS project breakdown, by implementation stage and status,December quarter 2017Note: Agencies apply RAG statuses at different implementation stages, so not all projects reportedon the ICT Dashboard have a RAG status.Source: VAGO, based on the data files available from data.vic.For this audit, we reviewed eight of these DHHS projects in detail.For the projects we reviewed, DHHS could not verify all the data reportedon the ICT Dashboard and we experienced substantial difficulty in obtainingdocumentation for some data fields.We identified inaccuracies in the eight projects we examined at DHHS. Althoughsome of the inaccuracies are minor, there was a large number of data fieldswhere the documentation provided did not match published data, or sourcedocumentation could not be provided.DHHS told us that it had not had a consistent project management office (PMO)or project reporting tool. This has resulted in differing maturity of practices inproject status reporting and knowledge management.Appendix B has further details of our testing.10201 1InitiatedDeliveryPostponedClosed—completedClosed—mergedClosed—terminated194GreenAmber46 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportDepartment of Premier and CabinetDPC has reported a total of six projects on the ICT Dashboard since its launch.For the December quarter 2017, DPC reported four ICT projects with a totalplanned expenditure of $102.5 million. The implementation stage and RAGstatus of these projects are shown in Figure 3C.Figure 3CDPC project breakdown, by implementation stage and status,December quarter 2017Note: Agencies apply RAG statuses at different implementation stages, so not all projects reportedon the ICT Dashboard have a RAG status.Source: VAGO, based on the data files available from data.vic.We reviewed one project in detail at DPC, which is also designated as an HVHRproject.DPC provided us with most of the information we needed to verify project datareported on the dashboard. We observed, however, that all of the project’scosts were reported as delivery costs, when, under the ICT Reporting Standard,initiation costs should have been separately reported.13InitiatedDeliveryPostponedClosed—completedClosed—mergedClosed—terminated21GreenAmberVictorian Auditor‐General’s Report The Victorian Government ICT Dashboard 47Melbourne WaterSince the ICT Dashboard’s launch, MW has reported 33 projects.MW is currently reporting 13 projects with a planned expenditure of$16.7 million, as shown in Figure 3D.Figure 3DMW project breakdown, by implementation stage and status,December quarter 2017Note: Agencies apply RAG statuses at different implementation stages, so not all projects reportedon the ICT Dashboard have a RAG status.Source: VAGO, based on the data files available from data.vic.MW provided us with documentation to verify the majority of the data reportedon the ICT Dashboard.We found six inaccuracies in MW’s projects’ revised completion dates and costs.These inaccuracies do not materially detract from the time and cost reporting ofMW’s ICT projects.553InitiatedDeliveryPostponedClosed—completedClosed—mergedClosed—terminated71GreenAmber48 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportPublic Transport VictoriaPTV has reported a total of 20 projects on the ICT Dashboard since its inception.In the December quarter 2017, PTV reported 10 projects with a plannedexpenditure of $89.9 million.Figure 3E shows the breakdown of PTV’s projects by implementation stage andstatus.Figure 3EPTV project breakdown, by implementation stage and status,December quarter 2017Note: Agencies apply RAG statuses at different implementation stages, so not all projects reportedon the ICT Dashboard have a RAG status.Source: VAGO, based on the data files available from data.vic.PTV provided us with documentation to verify the majority of the data reportedon the ICT Dashboard. We found that reported dollar figures were mostlyaccurate, although, in one case, the amount was rounded up from an internalestimate of $9.3 million to $10 million.PTV could not provide us with documentation to verify some data fortwo projects that were approved and commenced under the formerDepartment of Transport.PTV told us that this was due to limited documentation being handed overwhen the former Department of Transport transferred the projects to PTV.Consistency of interpretation and categorisationThe ICT Reporting Standard and ICT Expenditure Reporting Guideline for theVictorian Government provide guidance on how agencies should compile theirdata for reporting on the ICT Dashboard, and to comply with FRD 22H.1612InitiatedDeliveryPostponedClosed—completedClosed—mergedClosed—terminated81GreenAmber3.4 Do agenciesconsistently applythe guidance?Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 49The agencies we audited are not consistently applying the ICT ReportingStandard. We noted different interpretations by agencies of the criteria theyshould use to determine their overall RAG status, allocate internal staff coststo projects, and select a project category.Application of RAG statusAgencies we audited were aware of the ICT Reporting Standard, althoughsome of their staff were not overly familiar with its details, especially therequirements around allocating a RAG status.We observed that some agencies were not aware that, under version 1.0 ofthe ICT Reporting Standard, the RAG status was solely based on the project’sperformance against schedule. When these agencies reported a RAG status,they had based it on the overall status of the project or a combination of theschedule and budget status.Under version 2.0 of the ICT Reporting Standard, the RAG status is now basedon the overall status of the project, as reported in the most recent project statusupdate to the project control board (or equivalent) within the reporting agency.We observed during fieldwork that MW, PTV and some staff at DHHS were notaware of version 2.0 of the ICT Reporting Standard.We also observed that each agency uses a slightly different set of criteria todetermine its overall RAG status, as shown in Figure 3F.Figure 3FCriteria used to determine overall RAG status
CriteriaBudgetScheduleScopeRisksIssuesResourcesZero harmDeliverablesBenefitsKey stakeholdersHVHRChange requests
(a) Criteria according to DHHS official project status reporting template, however some projectshave established their own criteria for the RAG status.(b) Criteria used by the DPC project we reviewed. DPC does not have a standardised RAG reportingcriteria.Source: VAGO, based on information provided by agencies.50 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportThis means that while agencies may be accurately reporting the project’s RAGstatus on the ICT Dashboard, it is not possible to compare RAG status betweenagencies due to the different criteria used.We expected to find that agencies would have one set of criteria used todetermine the overall RAG status. While not all agencies use the same criteria,we expected that the RAG criteria would be consistent within agencies. Wefound this was not the case at DHHS, as outlined in Figure 3G.Figure 3GVariances in determining RAG status in DHHSDHHS’s PMO has been disbanded and re‐formed a number of times. Most recentlythe PMO was re‐established in late 2016.One of the consequences of this is that projects have established their own projectreporting mechanisms, including RAG status.Of the eight projects we examined at DHHS, we found that: one project had no formally defined criteria for determining the RAG status four projects used the PMO’s criteria, as outlined in Figure 3F (scope, schedule,budget, risk, resources and issues) one project used scope, schedule, budget and resources one project used scope, schedule, budget, benefits, dependencies andgovernance one project used scope, schedule, quality and risk/issues.DHHS is in the process of implementing a project management tool that is intended tofacilitate consistency in RAG status and project reporting.Source: VAGO, based on information from DHHS.Cost componentsWe observed that agencies use different practices when accounting forongoing employees within ICT project costings. At MW and PTV, staff costs andoverheads for public sector staff who work full‐time on projects are allocatedto the project’s cost, but at DHHS and DPC they are not. DHHS advised us thatoccasionally staff costs are allocated to projects, however, it does not requireactivity‐based costing which would give a more accurate allocation of costs.In all agencies, contractor costs directly attributable to a project were allocatedto the project.ICT system descriptionThe ICT Reporting Standard requires that agencies select a project category fortheir project. Agencies have 31 categories to choose from and may select morethan one category.Examples of categories include ‘customer relationship management’, ‘supplychain management’, ‘analysis’, ‘statistics’ and ‘other’. The ICT ReportingStandard refers agencies to the Australian Government Architecture frameworkfor assistance in selecting categories.Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 51Currently, there is limited oversight or quality assurance by DPC of the projectcategories selected by agencies. DPC only intervenes when a project type of‘other’ is selected, and then assists the agency to select one, or more, of theremaining 30 categories.For example, we observed that DHHS and the Environment Protection Authoritywere implementing very similar projects—an electronic document and recordsmanagement system. The Environment Protection Authority selectedthree categories for this project—‘document management’, ‘knowledgemanagement’, and ‘records management’—while DHHS only selected onecategory, ‘records management’.In another example, seven hospitals and health services are implementingelectronic medical record projects. Figure 3H shows the different categoriesselected by each agency for the project.Figure 3HProject categories selected for electronic medical record projectsProjectCustomerrelationshipmanagementRecordsmanagementManagementof processKnowledgemanagementDevelopmentand integration
Project 1Project 2Project 3Project 4Project 5Project 6Project 7
Source: VAGO, from information from the ICT Dashboard.While none of the selections in these examples is incorrect, they illustrate thatif DPC is relying on this self‐reported information to conduct analysis of ICTactivity and spend in certain categories, then DPC will need to assure itselfabout the quality and appropriateness of project category selections byagencies.Implications for accuracyIn most of the agencies we audited, the level of inaccuracies or errors wedetected did not materially undermine the overall accuracy of data reportedon the ICT Dashboard by that agency.However, the two projects at DHHS where we could not verify most of the datafields were an issue. These projects are detailed in Figures 3I and 3J.52 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportFigure 3IHousing Infrastructure Capital RefreshThe Housing Infrastructure Capital Refresh project is replacing ageing andunsupported hardware for the Housing Integrated Information Program. This is acomputer system used to deliver housing services to Victorians.The project commenced in May 2015 and is valued at $5.4 million.Based on the available information, we could not verify the project’s: initiation phase end date of 13 December 2016 initiation cost of $415 000 revised delivery dates of 30 June 2017 and 30 May 2018 revised delivery cost of $5.1 million.DHHS was able to provide us with documentation detailing the start date and planneddelivery end date. DHHS also provided us with some documentation around theplanned delivery cost of $5 million, however, we found inaccuracies in the figureprovided to us.Source: VAGO, based on information provided by DHHS.If the level of inaccuracy described in Figure 3I also exists in projects or agenciesthat we did not examine for this audit, it would raise concerns about the overallaccuracy of the ICT Dashboard.We found that agencies have varied processes to identify and report on ICTprojects valued at more than $1 million that should be reported on the ICTDashboard.Identifying projects over $1 millionAgencies have varied processes to identify projects valued at over $1 million.No agency has a fully automated system for identifying all ICT projects worthover $1 million.DHHS and MW use an automated search of their project management tool toidentify projects. PTV uses a manual process to identify projects. DPC does nothave a formal process for identifying projects that need to be reported. All theagencies that we audited manually verify the projects reported in the previousquarter.MW has an automated process to identify all projects valued at over $1 million,but still missed one project that should have been reported on the ICTDashboard.This shows that having a partially automated system is one control to assistagencies to identify all ICT projects for reporting, but it should not be the solemethod used to identify relevant projects.3.5 Are agencysystems andprocessesadequate?Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 53Reporting on projectsAgencies have varied processes for how they internally collate information forreporting on the ICT Dashboard. Agencies generally either have the projectmanagement team collate status reporting or have individual project managerscomplete the data update.At MW and PTV, the project management or reporting team compiles thereport, with quality assurance provided by program managers. At MW, externalquality assurance is also provided by the finance team. In DHHS, the projectmanagers compile the report and the PMO team provides oversight and qualityassurance.We found that all agencies have established internal approval and qualityassurance processes. ICT Dashboard reporting is approved by a suitably seniorofficer, after a series of data checks.Knowledge management and record keepingKnowledge management and record keeping are critical to maintainingcorporate and project‐specific knowledge. Previous audits have highlightedknowledge management and record keeping challenges across the public sector.We found that agency record keeping and knowledge management processeswere generally acceptable, but were more problematic when it came to olderprojects that had been transferred between agencies, as shown in Figures 3Jand 3K.Sound record keeping and knowledge management processes were lacking atDHHS.Figure 3JVictorian Health Incident Management SystemThe Victorian Health Incident Management System project involves improving thesystem’s functionality, reporting process and data analysis capability.DHHS advised that the project commenced in July 2015. In 2017, the Victorian Agencyfor Health Information (VAHI), an administrative office under DHHS, took overresponsibility for the project.Based on the available information, we could not verify the project’s: reported start date of 6 July 2015 initiation phase end date of 1 January 2017 initiation cost of $2.25 million planned implementation cost of $2.7 million RAG status.VAHI advised that there was limited documentation transferred when the projectmoved from DHHS to VAHI, including the project’s initial approval. VAHI reported$2.25 million and $2.7 million as the initiation and delivery costs, respectively.However, we found that the $2.25 million was not for initiation activities as definedunder the ICT Reporting Standard. Rather, this amount was the cost incurred prior toVAHI taking on the project, which was for delivery activities.Source: VAGO, based in information provided by DHHS and VAHI.54 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportFigure 3KRegional Radio Communications NetworkThe Regional Radio Communications Network project involves the modernisation ofthree legacy train communications systems.Based on available information, early works for the project commenced sometime in2011, and the business case was completed in early 2013. The former Department ofTransport completed both the early works and business case. PTV took overresponsibility for the project in April 2012.Based on the available information, we could not verify the project’s: reported start date of 1 March 2012 initiation cost of $7.25 million.PTV was able to provide us with some information regarding the start date andpre‐implementation costs, however, we were unable to identify the sourcedocumentation for either field or how the figures were determined.Source: VAGO, based in information provided by PTV.Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 55Appendix AAudit Act 1994 section 16—submissions and commentsWe have consulted with DHHS, DPC, DTF, MW and PTV, and we considered theirviews when reaching our audit conclusions.As required by section 16(3) of the Audit Act 1994, we gave a draft copy of thisreport to those agencies and asked for their submissions and comments.Responsibility for the accuracy, fairness and balance of those comments restssolely with the agency head.Responses were received as follows:DHHS …………………………………………………………………………………………………….. 56DPC ………………………………………………………………………………………………………. 59DTF ……………………………………………………………………………………………………….. 62MW ………………………………………………………………………………………………………. 63PTV ……………………………………………………………………………………………………….. 6456 The Victorian Government ICT Dashboard Victorian Auditor-General’s ReportRESPONSE provided by the Secretary, DHHSVictorian Auditor-General’s Report The Victorian Government ICT Dashboard 57RESPONSE provided by the Secretary, DHHS—continued58 The Victorian Government ICT Dashboard Victorian Auditor-General’s ReportRESPONSE provided by the Secretary, DHHS—continuedVictorian Auditor-General’s Report The Victorian Government ICT Dashboard 59RESPONSE provided by the Secretary, DPC60 The Victorian Government ICT Dashboard Victorian Auditor-General’s ReportRESPONSE provided by the Secretary, DPC—continuedVictorian Auditor-General’s Report The Victorian Government ICT Dashboard 61RESPONSE provided by the Secretary, DPC—continued62 The Victorian Government ICT Dashboard Victorian Auditor-General’s ReportRESPONSE provided by the Secretary, DTFVictorian Auditor-General’s Report The Victorian Government ICT Dashboard 63RESPONSE provided by the Managing Director, MW64 The Victorian Government ICT Dashboard Victorian Auditor-General’s ReportRESPONSE provided by the Chief Executive Officer, PTVVictorian Auditor-General’s Report The Victorian Government ICT Dashboard 65RESPONSE provided by the Chief Executive Officer, PTV—continued
Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 67Appendix BDetailed results fromagency testingTo assess the accuracy and completeness of the ICT Dashboard, we examinedsource data from projects reported by four of the agencies involved in thisaudit: DHHS DPC MW PTV.We tested the information published on the ICT Dashboard for 18 ICT projects,and asked these agencies to provide source data or documentation to verifytheir published data.Figure B1 shows a summary of the results from our detailed audit testing.68 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportFigure B1Results of agency testingProject Start dateInitiationend dateInitiationcostPlanneddelivery enddatePlanneddelivery costReviseddelivery costDecemberquarter2017–18RAG statusDHHSFeedbackManagementSystem (a) (b) (c)HousingInfrastructureCapital Refresh (d) (d) (e) (d) (d)InformationSharing Gateway (b) Modelling andPaymentsSystem (f) N/A (g)PersonalHardshipAssistancePrograms (h) N/A (c)VictorianeReferralProgram (a) N/A Victorian HealthIncidentManagementSystem(i) (i) (j) (j) N/A (d)Windows ServerRemediationProgram DPC
DeveloperFunded WorksSystem (l) Site SecurityUpgrades (l) SCADA MosaicUpgrade (l) Website—Digital CustomerExperiencePlatform (l) Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 69Figure B1Results of agency testing—continuedProject Start dateInitiationend dateInitiationcostPlanneddelivery enddatePlanneddelivery costReviseddelivery costDecemberquarter2017–18RAG statusPTVBus InformationImprovementProject(f) N/A OperatorPayment SystemModernisation (m) (n) N/A(o)Regional RadioCommunicationsNetwork(i) (i) N/A Train SCADA(Electrol)Replacement (i) (i) N/A(o)VehiclePassengerInformationSystem A & CClass Key: Accurately verified; verified with minor errors; large errors; could not be verified; N/A means the agency has not needed tocomplete this field.(a) Error due to agency publishing a planned date, not actual date.(b) Discrepancy unable to be explained by agency.(c) Agency used different criteria to determine RAG status, rather than criteria under ICT Reporting Standard, version 2.0.(d) No documents available to verify the data.(e) Large difference between published data and documentation provided by agency to verify the data.(f) Agency advised that this inaccuracy was due to a translation or typographical error.(g) DHHS internal project status report reported the status as half green, half yellow, but reported green on the ICT Dashboard.(h) Cost not reported.(i) Unable to verify due to a lack of project documentation being handed over when the project transferred between responsible agencies.(j) Agency reported delivery costs as initiation costs.(k) DPC reported all costs as delivery costs, including initiation costs, due to the challenges of determining the split between initiation anddelivery phases when using the Agile methodology.(l) Agency reported all costs as delivery costs, including initiation costs.(m) PTV advised this error was due to the selection of a point in time milestone date being reported, and then the cell was locked down.(n) PTV advised this inaccuracy was due to a rounding error of $700 000.(o) Project was completed in Q1 2017–18, so did not report in Q2 2017–18.Source: VAGO assessment of information provided by audited agencies.
Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 71Appendix CICT project reportingrequirementsDuring the audit, we conducted detailed testing of a selection of agencies, andprojects reported on the ICT Dashboard. Figure C1 shows the ICT projectreporting requirements for PR under version 1.0 of the ICT Reporting Standard.We undertook detailed testing using version 1.0 of the ICT Reporting Standard.Figure C1ICT PR requirements, under version 1.0 of the ICT Reporting Standard
RequirementAgency nameGovernment domain
DetailThe name of the public body. Business and Industry Community Services
Culture, Sport and Recreation Education Emergencies and Safety Employment and Workplace Environment and Water Government and Economy Health Law and Justice Property and Planning Tourism Transport and Vehicles
Project titleDescriptionStart dateProject managementmethodology used
The name of the project for reporting purposes.Short description and objectives of the project.The start date or day one of the pre‐implementation activities of the project.
Primary project management methodology used/to be used to manage theproject (select one): PMBoK PRINCE2 Agile Other—please specify.72 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportFigure C1ICT PR requirements, under version 1.0 of the ICT Reporting Standard—continued
RequirementCurrent project stage
DetailThe current project stage as defined below (select one): Initiated—includes projects in feasibility, planning or business casedevelopment phases, including those that never progressed beyond this
phase and those that may have been postponed In progress—pre‐implementation activities In progress—implementation activities Project completed—project closed Project postponed Project terminated (prior to completion).
Pre‐implementationstage end date(a)
Project end date for all pre‐implementation activities.PR‐09 Pre‐implementationcostThe cost of all pre‐implementation activities of the project.PR‐10 Plannedimplementationend date(b)The planned implementation completion date for the project that was estimatedon completion of pre‐implementation phase. This field is not required until theproject enters the implementation stage but once entered remains set for theremainder of the project.PR‐11 Plannedimplementation costThe planned implementation expenditure for the project that was estimated oncompletion of pre‐implementation phase. This field is not required until theproject enters the implementation stage but once entered remains set for theremainder of the project.PR‐12 Revisedimplementationend dateThe revised implementation end date for the project if this changes from theplanned implementation date.PR‐13 Revisedimplementation costThe revised implementation expenditure for the project if this changes from theplanned implementation cost.PR‐14 ImplementationstatusStatus of the current implementation timeline: Green (G)—time line is on track Amber (A)—time line issues exist but they are under the project manager’scontrol Red (R)—serious time line issues exist that are beyond the project manager’scontrol. They require assistance to resolve. The time line is likely to change.PR‐15 PreviousimplementationstatusImplementation project status for the last 12 months: Last reporting period – 3 months ago Reporting period 6 months ago Reporting period 9 months ago Reporting period 12 months ago.PR‐16 Reporting periodcommentsGeneral comments for the current reporting period including an explanation ofany project variances.PR‐17 Date at which datawas currentThe date of the information used to compile this report.(a) Under version 2.0 of the ICT Reporting Standard, pre‐implementation was re‐named ‘initiation’.(b) Under version 2.0 of the ICT Reporting Standard, implementation was re‐named ‘delivery’.Source: VAGO, based on version 1.0 of the ICT Reporting Standard.Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 73Version 2.0 of the ICT Reporting Standard altered the fields that are reported onthe ICT Dashboard. Figure C2 shows the data fields under version 2.0 of the ICTReporting Standard.Figure C2ICT PR requirements, under version 2.0 of the ICT Reporting Standard
New data fieldsAgency nameGovernment domainICT project category
DetailRefer to Figure C1 PR‐01.Refer to Figure C1 PR‐02. Customer Relationship Management Customer Preferences
Customer Initiated Assistance Tracking and Workflow Routing and Scheduling Management of Process Organisational Management Investment Management Supply Chain Management Content Management Document Management Knowledge Management Records Management Analyses and Statistics Visualisation Knowledge Discovery Business Intelligence Reporting Data Management Human Resources Financial Management Assets/Materials Management Development and Integration Human Capital / Workforce Management Security Management Collaboration Search Communication Systems Management Forms Management Other—please specify
The name of the project as per the business case.Refer to Figure C1 PR‐04.
74 The Victorian Government ICT Dashboard Victorian Auditor‐General’s ReportFigure C2ICT PR requirements, under version 2.0 of the ICT Reporting Standard—continued
New data fieldsProject managementmethodology used
DetailRefer to Figure C1 PR‐06.
PR‐07 Current project stage The current project stage as defined below (select one): Project Initiated—pre‐‘Project Delivery’ activities are underway includingpreliminary planning, feasibility study, business case development and/orfunding request. Business Case Approved—project business case has been approved orfunding has been allocated. Project Delivery—project has commenced delivery. Project Completed—project has delivered its outcomes, and is being or hasbeen closed. Project Postponed—project has been temporarily put on hold. Project Terminated prior to ‘Project Completed’—work on the project hasended prematurely (i.e., a decision has been made to stop work). Project Merged—scope of the project has been merged into another project.PR‐08 ‘Project Initiated’start dateThe start date or day one of the ‘Project Initiated’ activities.PR‐09 Planned ‘ProjectInitiated’ end dateThe planned end date for all ‘Project Initiated’ activities.PR‐10 Revised ‘ProjectInitiated’ end date(if applicable)The revised end date for all ‘Project Initiated’ activities if date differs from PR‐09.PR‐11 Planned ‘ProjectInitiated’ costThe planned expenditure for all ‘Project Initiated’ activities.PR‐12 Revised ‘ProjectInitiated’ cost(if applicable)The revised expenditure for all ‘Project Initiated’ activities if expenditure differsfrom PR‐11.PR‐13 ‘Business CaseApproved’ dateThe date when the business case was approved.PR‐14 Planned ‘ProjectDelivery’ start dateThe planned start date for all ‘Project Delivery’ activities specified on completionof the ‘Business Case Approved’ stage.PR‐15 Revised ‘ProjectDelivery’ start date(if applicable)The revised start date for all ‘Project Delivery’ activities if date differs fromplanned ‘Project Delivery’ start date.PR‐16 Planned ‘ProjectDelivery’ end dateThe planned end date for all ‘Project Delivery’ activities specified on completionof the ‘Business Case Approved’ stage.PR‐17 Revised ‘ProjectDelivery’ end date(if applicable)The revised end date for all ‘Project Delivery’ activities if date differs from PR‐16.PR‐18 Planned ‘ProjectDelivery’ end costThe planned expenditure for all ‘Project Delivery’ activities specified oncompletion of the ‘Business Case Approved’ stage.PR‐19 Revised ‘ProjectDelivery’ cost(if applicable)The revised expenditure for all ‘Project Delivery’ activities if expenditure differsfrom PR‐18.Victorian Auditor‐General’s Report The Victorian Government ICT Dashboard 75Figure C2ICT PR requirements, under version 2.0 of the ICT Reporting Standard—continued
New data fieldsProject status
DetailOverall status of the project as per the most recent project control board report: Green (G)—on track
Amber (A)—issues exist but they are under the project manager’s control Red (R)—serious issues exist that are beyond the project manager’s control.
Project status for the last 12 months: Last reporting period – 3 months ago. Reporting period 6 months ago. Reporting period 9 months ago. Reporting period 12 months ago.PR‐22 Reporting periodcommentsGeneral comments for the current reporting period including an explanation ofany project variances, including plans to recover if the project status is amber(A) or red (R) as per PR‐20.PR‐23 Date at which datawas currentThe date of the information used to compile this report. The date should reflectthe current reporting period.PR‐24 Date projectcompleted,postponed,terminated or mergedThe date the project was completed, postponed, terminated or merged withanother project.Source: VAGO, based on version 2.0 of the ICT Reporting Standard.
Auditor‐General’s reportstabled during 2017–18Report title Date tabledV/Line Passenger Services (2017–18:1) August 2017Internal Audit Performance (2017–18:2) August 2017Effectively Planning for Population Growth (2017–18:3) August 2017Victorian Public Hospital Operating Theatre Efficiency (2017–18:4) October 2017Auditor‐General’s Report on the Annual Financial Report of the Stateof Victoria, 2016–17 (2017–18:5)November 2017Results of 2016–17 Audits: Water Entities (2017–18:6) November 2017Results of 2016–17 Audits: Public Hospitals (2017–18:7) November 2017Results of 2016–17 Audits: Local Government (2017–18:8) November 2017ICT Disaster Recovery Planning (2017–18:9) November 2017Managing the Level Crossing Removal Program (2017–18:10) December 2017Improving Victoria’s Air Quality (2017–18:11) March 2018Local Government and Economic Development (2017–18:12) March 2018Managing Surplus Government Land (2017–18:13) March 2018Protecting Victoria’s Coastal Assets (2017–18:14) March 2018Safety and Cost Effectiveness of Private Prisons (2017–18:15) March 2018Fraud and Corruption Control (2017–18:16) March 2018Maintaining the Mental Health of Child Protection Practitioners(2017–18:17)May 2018Assessing Benefits from the Regional Rail Link Project (2017–18:18) May 2018Results of 2017 Audits: Technical and Further Education Institutes(2017–18:19)May 2018Results of 2017 Audits: Universities (2017–18:20) May 2018Community Health Program (2017–18:21) June 2018All reports are available for download in PDF and HTML format on our websitewww.audit.vic.gov.auVictorian Auditor‐General’s OfficeLevel 31, 35 Collins StreetMelbourne Vic 3000AUSTRALIAPhone +61 3 8601 7000Email [email protected]
- Assignment status: Already Solved By Our Experts
- (USA, AUS, UK & CA PhD. Writers)
- CLICK HERE TO GET A PROFESSIONAL WRITER TO WORK ON THIS PAPER AND OTHER SIMILAR PAPERS, GET A NON PLAGIARIZED PAPER FROM OUR EXPERTS
QUALITY: 100% ORIGINAL PAPER – NO PLAGIARISM – CUSTOM PAPER