Tools for BIM maturity and benefits not fit for purpose, says major study

There is a lack of effective tools for assessing BIM maturity, or for evaluating its benefits, says a report commissioned from Northumbria University by the Centre for Digital Built Britain (CDBB) in partnership with the UK BIM Alliance.

The report’s authors examined 25 different ways of evaluating BIM maturity and benefits available to industry. This included 15 maturity tools, four “maturity” methods for organisations and projects, and six benefits tools.

None of the maturity assessment tools examined were aligned with ISO 19650, the international standard for managing information over the whole lifecycle of an asset, as they were all developed prior to its release in 2018. The report recommended that a framework be developed for maturity assessment tools, aligned with the ISO 19650 series, to ensure a common approach.

The study found that BIM benefits evaluation tools have value in helping to promote BIM and encourage collaborative working but recommended that these tools be extended to look at benefits throughout the whole supply chain.

With the launch of the ISO 19650 series in 2018, CDBB decided it was a sensible time to review the various tools used for assessing BIM maturity and measuring BIM benefits, to understand what the gaps were.

The research project team included UNN, BIM Academy, the Royal Institute of British Architects (RIBA) and HKA. The project was led by associate professor Mohamad Kassem (pictured above), who also led the authoring of the report with Jennifer Li of UNN. Contributing authors were Professor Bimal Kumar and Richard Watson of UNN, Adrian Malleson of RIBA, Dr Graham Kelly of BIM Academy and Dr David-John Gibbs of HKA.

The study found that BIM maturity assessment tools are not widely used. Only 28% of the 184 survey respondents used a tool to measure BIM maturity. Another 18% measured BIM maturity without a tool.

The research showed gaps between the characteristics of existing BIM maturity assessment tools and the industry requirements. Criticisms include:

  • The tools are typically rigid, with binary (yes/no) inputs from users and are largely focused on readiness and capability, rather than maturity;
  • The tools tend to lack granularity, which means the depth of the assessment is low or moderate. Where the granularity is higher, assessments take significantly longer to conduct;
  • A lack of precision limits the usefulness of some tools; there is insufficient distinction made between readiness, capability and maturity.
  • Maturity assessments do not reach far enough down into the supply chain, instead focusing on tier one contractors and lead designers;
  • The gaps and limitations in existing tools are driving organisations to develop their own internal maturity assessments. Some 45% of respondents had already done this, according to a survey conducted by the report’s authors;
  • The depth of assessment enabled by most tools is low, offering limited understanding of BIM maturity of organisations or projects;
  • Maturity assessment is currently geared towards complying with clients’ BIM requirements, but these differ from client to client.

The survey identified several positive reasons for measuring BIM maturity, including that it helps organisations identify their BIM implementation challenges and develop improvement strategies.

As with maturity assessment tools, the report found that BIM benefits evaluation tools are not commonly used. Just 16% of survey respondents used a tool to measure BIM benefits, with 35% saying they measure benefits without a tool, and 49% saying they do not measure benefits at all.

The task of measuring BIM benefits is complex, says the report: it is hard to eliminate alternative explanations for the relationship between a BIM capability and an end benefit; there is a lack of benchmarking data and there is variation in the knowledge of those inputting data. There is a further challenge in communicating benefits.  

If organisations are to adopt BIM benefits evaluation methodologies, they need to ensure that the benefits sought are material and relevant over long project lifespans.

Some 92% of survey respondents agreed that measuring BIM benefits encourages a collaborative way of working, while 77% agreed there is a need for better measurement tools.

Gaps identified by the report in the assessment of both BIM maturity and benefits evaluation tools need to be addressed, to fulfil the requirements and expectations of the construction and asset management industry.

The report’s key recommendations are:

  • All BIM maturity assessment tools should be aligned with the ISO 19650 series.
  • At an organisational level, a multi-level framework for effective BIM maturity assessment should be developed to provide a common approach within the industry. This should identify a range of BIM competencies required and propose metrics for their assessment. The framework should have one common level, relevant to all disciplines within construction and asset management, which is also adaptable to suit specific organisations. Additional levels should be specific to different disciplines.
  • At a project level, a BIM maturity assessment method should be developed based on the UK BIM Framework, including the ISO 19650 series.
  • BIM competencies, including maturity, should play a greater role at the “invitation to tender”, “tender response”, “appointment”, and “mobilisation” stages of a project. The competencies should be extended beyond readiness and capability to include maturity. This process can be assisted by adopting the ISO 19650 series which sets out enabling requirements throughout the tender process for each appointment.
  • BIM benefits evaluation tools should be extended to include the broader benefits of embracing a digital culture and driving digital maturity in the supply chain.
  • At a project level, the tools should establish metrics at the outset of a project and consistently measure against targets. These metrics need to address challenges that are commonly experienced with construction projects, including: lengthy project lifetime; time lag between an activity and its corresponding benefit; variation in a project’s scope; and the continuing evolution of technology and processes.

Image: Associate professor Mohamad Kassem (Northumbria University)

Story for BIM+? Get in touch via email: [email protected]


  1. Firstly can you add a link to the report to this article so everyone can read it for them selves?

    I am interested to understand what do you mean by BIM maturity?

    Also which 25 assessment tools there are?

    And finally I am in agreement that everything will need to align to ISO 19650 but it cannot until all have been written.

  2. I think the last comment is the only relevant question in the article. ‘At a project level, the tools should establish metrics at the outset of a project and consistently measure against targets.’, If we do not know what the problems are and the metrics associated with them then we do not have anything to measure against. A simple measurement that we used to use in design offices is ‘how long does it take to produce the production information for a given project, by facility type’. Even this question cannot be answered today because nobody measures it. Do cad (BIM) tools produce this information faster, once again we do not know. We believe what the vendors tell us; that the cad tools (BIM) produce the drawings faster, can anybody supply a research activity that has answered this simple question?? That’s just the simple bit.

  3. Seems like the report would be an interesting read, although I’m not sure all of the positions are valid, based on how this article reads. On the one hand the report is critical of binary inputs within the Maturity tools and then on the other recommends that all BIM Maturity tools align with a single standard.
    An international standard is not ‘best’ practice. A standard helps achieve consistency, but is often a lowest common denominator.
    We develop best practice by learning what works well and what doesn’t, based in what is needed. Preparedness to fail, success and innovation are all linked.

  4. Dear Gareth,

    My research address your question in previous comment. Feel free to write me at [email protected] for the article or thesis.

    My research is focusing on how to establish organisation or project bim strategy – define vision, mission, goals, objectives, success factors and performance measures.

    In fact We are now implementing it at India’s biggest contractor Organisation and planning to measure the performance.

    We had compared Different maturity assessment methods prior to this research publish. And, made a comprehensive list of Success factors for BIM Projects.

    URL for article

    Dr Amarnath CB
    Head BIM Strategy, L&T Const.

  5. Dear Gareth,

    Thank you for your comments. The answers can be found in the report, which can be downloaded here:

    Kind regards,
    Dr David-John Gibbs

  6. The fundamental principles of BS1192 and ISO 19650 are pretty much the same, so although the BIM maturity levels are not present in the ISO 19650, the tools used to assess the level of maturity within an organisation, are still very much fit for purpose. We, therefore, should not get too hung up on which standard they align to. After all, we are in a period of transition.

  7. This is a useful article, but for the uninitiated, it’d be helpful to have some definitions of key terms. I’ve been pointed to page 19 of the full report which DJ has helpfully included a link to. That provides a useful glossary.

Comments are closed.

Latest articles in News