AIArtificial intelligence

AI projects: who measures the results, and by what rules

In a structured organisation, measuring an AI project is not left to whoever proposed it. Management accounting takes it on, with rules approved in advance, comparisons between sites, agreements on logging and indicators for the board.

Strategic editorial team 2 October 2026 8 min read

Cover: “AI projects: who measures the results”, Artificial intelligence

In a structured organisation, the results of an artificial intelligence project are measured by a different function from the one that proposed it. That is usually management accounting, the team that runs budgets and performance reporting. It works to a measurement protocol approved by management before launch. The protocol sets out which indicators count and how they are calculated, and which sites or departments the AI users are compared with. It also states which data may be recorded under Italy’s Workers’ Statute (Statuto dei lavoratori) and which figures go to the board. It is the same separation the organisation already applies to budgets and investments. Whoever asks for the resources does not certify the return on their own.

Why shouldn’t measurement be left to whoever proposed the project?

The use of artificial intelligence is growing fast. According to Eurostat, 20% of EU enterprises with at least ten employees used AI technologies in 2025, up from 13.5% the year before. The effects on the bottom line are harder to prove. In McKinsey’s survey published in November 2025, with almost two thousand participants, 88% said their organisation regularly uses AI in at least one business function. Only 39% reported any effect on operating profit (EBIT) for the organisation as a whole.

In between lies a conflict of interest that no one declares. The department that wanted the tool, and the vendor that sold it, both have an interest in reading the data in the most favourable way. They choose the period, leave out the unusual cases, and count the hours saved but not the hours spent on corrections. No bad faith is needed. Confidence in one’s own project is enough. The countermeasure is organisational. Whoever proposes describes the expected benefits. Whoever measures checks them, under rules that cannot be changed halfway through.

Who does what?

Function	Role in the measurement
Management	Approves the measurement protocol and decides whether to continue, adjust or stop
Management accounting	Defines how the indicators are calculated, collects the data and writes the report
Process owner	Describes the expected benefits and explains anomalies, without writing the report
IT	Provides extracts from the systems and makes sure the logs are complete
HR and legal	Check that the data collected complies with employment and data protection rules
Internal audit, where it exists	Spot-checks that the method has been followed

The measurement protocol is a short document. It lists the indicators with their formulas, the source of each data item, the baseline period before launch, the review dates and the thresholds for the three possible decisions. Once approved, it can be changed only through a new approval, with the reason recorded.

How do you use sites as a comparison?

The most robust comparison is with people who do the same work, over the same period, without AI. An organisation with several sites or departments has an advantage a small business lacks. It can launch the tool at one site and use the others as a comparison group. Or it can roll it out in stages, one site after another, and see whether the improvement follows each launch. If the improvement appears everywhere at the same moment, it probably comes from something else. It might be the end of a seasonal peak, a changed procedure or a new ERP system.

For the comparison to hold, a few conditions are needed.

The same indicators, calculated with the same formula, at every site
A baseline period before launch that covers at least one full work cycle, such as a month-end close
Every other change at the sites involved, recorded with its date
Pilot sites chosen because they are representative, not because they are the most enthusiastic

Time spent checking and correcting the AI’s output counts as working time for the process. It is the item that most often disappears from optimistic reports.

What data can be recorded about people’s work?

You measure the process, not the individual. In Italy, Article 4 of the Workers’ Statute (Law 300/1970) allows tools that could also be used to monitor employees’ activity remotely only for organisational and production needs, workplace safety and the protection of company assets. Except in the cases the same article provides for, it also requires an agreement with the workplace union representatives or, failing that, authorisation from the National Labour Inspectorate (Ispettorato nazionale del lavoro). Law 132/2025 on artificial intelligence further requires employers to inform workers when they use fully automated decision-making or monitoring systems. That duty follows Legislative Decree 152/1997, which governs the information employers must give staff, and leaves the Workers’ Statute fully in force. The GDPR requires that only the data needed for the purpose be collected.

In an organisation with union representatives, it is wise to bring the measurement protocol to the table before launch, together with the list of data the system records. An agreement that specifies the purpose, data aggregated by group, retention periods and a ban on individual assessments protects the company. It also makes people more comfortable using the tool. Those who fear being watched use it less, and the data suffers. For San Marino companies the framework is different and needs to be assessed case by case.

A few data items per transaction are enough. They are the date and time, the type of case, the outcome of the AI’s proposal (accepted, changed or rejected), the time to confirmation and any error report. Full conversation content would increase the personal data held without improving the measurement. The content and retention of the logs follow the criteria described in our article on how to connect an AI assistant to company systems.

What goes to the board?

The board receives one page, with the same structure at every review. The AI Risk Management Framework from NIST, the US standards body, devotes one of its four functions to measurement, alongside Govern, Map and Manage. Performance is measured, but so are risks and effects on people. For a board, this translates into four groups of figures.

Group	Example indicators
Value	Difference between pilot and comparison sites on time, volumes and costs, net of checking hours
Quality	Share of proposals accepted without changes, rework, related complaints
Risk	Wrong answers on sensitive topics, incidents, exposed data, permission exceptions
Adoption	People using the tool each week, drop-outs, user reports

Three lines go next to the figures. They state the proposed decision, the conditions behind it and the date of the next review. The report is signed by whoever measures, not by whoever proposed the project.

When do you decide to continue, adjust or stop?

The decision comes on the dates set in the protocol, not when it is convenient. The thresholds are written in advance and cover all four groups. A project that improves value but breaches the risk threshold does not carry on as if nothing had happened. The same rules apply to AI agents working inside company systems, where the testing figures determine when to extend the tasks assigned.

A typical scenario. A group with three sites introduces an assistant for internal questions to HR and the IT help desk. Management accounting writes the protocol. It tracks time to close requests, reopened requests, wrong answers on HR topics and weekly use. The agreement with the union representatives rules out individual assessments. The assistant goes live at one site, and the other two act as the comparison for two months. The report notes that the renewal of company benefits falls in the same period. The board receives one page. Times are down at the pilot site and stable at the others, reopened requests are unchanged, and two wrong answers stay below the threshold. The second site starts with indicators that have already been tested.

In brief

An AI project is measured by a different function from the one that proposed it, usually management accounting.
A protocol approved in advance sets indicators, formulas, sources, dates and thresholds, and it is not changed halfway through.
Sites or departments without AI, or a staged rollout, separate the tool’s effect from other changes.
The data recorded about work is agreed in advance, in line with the Workers’ Statute, Law 132/2025 and the GDPR.
The board receives one page on value, quality, risk and adoption, signed by whoever does the measuring.

Frequently asked questions

What if the organisation has no formal management accounting function?

The task can go to the finance department or to an external adviser who took no part in choosing the tool. What matters is the separation between whoever proposes and whoever measures, not the name of the office.

Can the vendor supply the data for the measurement?

It can supply technical extracts, such as usage logs, but calculating the indicators and writing the report remain with the organisation. The contract should provide for full access to the logs and specify their format.

Is a union agreement needed even for an assistant used only in the offices?

It depends on what the system records and how the data can be used. The assessment should be made with your lawyer, case by case. If the system makes it possible, even in theory, to reconstruct what individual people do, the Article 4 question arises.

How often should the measurement go to the board?

It goes on the dates set in the protocol, usually at the end of the trial and then at the same intervals as other periodic reports. A project that changes sites or tasks goes back to the board before it expands.

Writing the measurement protocol and separating whoever proposes from whoever verifies is part of the work of our artificial intelligence for organisations team. When a vendor has already proposed a project, the same approach becomes part of our second opinion on AI projects and vendors.

Sources

Consulted on 16 September 2026.

Find out moreArtificial intelligence for businessesArtificial intelligence built into the systems your company already uses. We choose the right processes, connect the data, and settle security and rules first.