AI: Evidence before approvals

Purpose

This briefing sets out evidence questions for boards considering artificial intelligence and advanced automation. Australian energy and resources operations provide the evidence examined here. The six tests are proposed governance tools, not statutory mandates. They are intended to help directors distinguish a promising demonstration from a system sufficiently understood, supported, and controlled for its proposed operating role.

Executive Summary

Australian evidence shows that defined automation and AI-related applications can deliver reported operational outcomes at specific sites. Those outcomes do not establish that benefits transfer reliably between assets, or that broad deployment is superior to a controlled, use-case-specific approach. The public sources reviewed for this briefing do not provide a comparable, sector-wide portfolio dataset showing which systems were attempted, sustained, modified, withdrawn, or decommissioned, alongside their full lifecycle costs and independently tested results. That finding is limited to the sources reviewed. It does not establish that such information is absent across the industry.

Boards should require evidence proportionate to a system's intended influence on operations and the consequences of failure. Approval should rest on a defined production boundary, representative performance, whole-of-life economics, safety controls, continuing assurance, and demonstrated workforce capacity. The staged decision pathway in Section 7 applies that principle. It is a proportionate governance practice, not a statutory safe harbour.

1. Why Boards Face This Question Now

Proposals for AI-enabled and automated systems now reach boards across many sectors. They are commonly framed in terms of productivity, safety, reliability, or cost. Much of the publicly available material describing these systems originates with operators, technology developers, funding agencies, or research institutions that participated in the work. Such material is valuable for identifying what has been deployed and what outcomes were reported. It is not, by itself, an independent evaluation, and it seldom discloses the projects that were discontinued, modified, or withdrawn.

The governance deficit is therefore an evidence deficit as much as a technology question. The sources reviewed for this briefing do not supply the numerators and denominators needed to calculate how many AI or automation pilots in Australian resources or energy operations became sustained production systems. Claims that most AI pilots fail to scale are equally difficult to rely upon for Australian decisions unless their samples, industries, geographies, and definitions are disclosed. Boards are therefore asked to decide on proposals whose supporting evidence is often uneven, selective, or incomplete.

Market and regulatory expectations are moving in a consistent direction, even where binding rules remain unsettled. The United States National Institute of Standards and Technology describes its AI Risk Management Framework as voluntary and lifecycle-oriented, treating governance, mapping, measurement, and management of AI risk as continuing activities (National Institute of Standards and Technology, 2023). Standards Australia adopted AS ISO/IEC 42001:2023, an AI management-system standard, on 16 February 2024 (Standards Australia, 2024). Neither instrument creates a general Australian legal obligation on its own, a point examined in Section 6. Their practical significance is that they describe what a reasonable governance process for AI can include.

Regulatory review in an adjacent sector points to the same governance risk. The Australian Securities and Investments Commission reviewed AI governance among financial services and credit licensees and reported cases in which governance arrangements lagged deployment (Australian Securities and Investments Commission, 2024). That review concerned financial services, not energy or resources operations, and its findings are cited here as a comparator for board oversight rather than as evidence about mining or energy assets.

Board oversight is the point at which these deficits become a decision. A director is not expected to be a specialist in machine learning. A director is expected to ask whether the evidence before the board is adequate for the decision in hand, and whether management has defined the conditions under which the system may operate, be monitored and be stopped. Sections 2 and 6 set out the legal framework within which that expectation sits.

2. The Director's Legal Duty of Inquiry

The phrase "duty of inquiry" is used in this briefing as a governance description. It is not statutory language. The statutory source is the duty of care and diligence in section 180 of the Corporations Act 2001 (Cth) (Commonwealth of Australia, 2001).

Section 180(1) requires a director or officer to exercise their powers and discharge their duties with the degree of care and diligence that a reasonable person would exercise in the same position and with the same responsibilities. That standard is assessed in the circumstances of the corporation and the office. A board that approves a material AI or automated system without enquiring into its evidence, its operating boundary or its fallback arrangements may find it difficult to show that the decision was made with reasonable care and diligence. This statement is an interpretation of the provision in general terms and must be verified against the current text and case law before reliance.

Section 180(2) contains a business judgment rule. In general terms, a director who makes a business judgment in good faith, for a proper purpose, without a material personal interest, on an informed basis and with a rational belief that the judgment is in the corporation's best interests, is taken to meet the statutory standard in relation to that judgment. The informed-basis requirement is the point at which evidence becomes relevant. The statutory test does not require a director to be correct. It requires a process that is reasonable and documented. Whether a particular evidence base satisfies that requirement is a question of fact and law for specific circumstances. This briefing does not answer it.

Officers also carry duties under work health and safety legislation. Section 27 of the Work Health and Safety Act 2011 (Cth) requires an officer to exercise due diligence to ensure the person conducting a business or undertaking complies with its duties (Commonwealth of Australia, 2011). The application of that provision in each jurisdiction is addressed in Section 6.

The relationship between these duties and AI decisions is practical rather than doctrinal. Where a board relies on a vendor's assurance, a management summary or a single pilot result, the question is whether the board understood what that material did and did not establish. The six tests in Section 5 are designed to make that understanding explicit. They are governance standards, not a checklist that discharges any statutory obligation.

3. What the Australian Evidence Shows

CSIRO's longwall automation case concerns a hardware-integrated automation system, not generic machine learning, or AI as a category. CSIRO describes specialised remote guidance technology that automatically plots equipment position in three dimensions. It reports adoption across approximately 60% of operating Australian longwall coal mines, with productivity improvements of 5 to 10% (CSIRO, 2019). The case is therefore best read as evidence for a defined engineering system with a measured operational outcome. It is not a general measure of how AI performs in mining.

The Agnew Renewable Energy Microgrid demonstrates an integrated engineering system at a remote Western Australian mine. It combines wind, solar, battery storage and gas generation with forecasting and energy-management controls. ARENA's project page describes renewable penetration of up to 60% as a forecast. Separately, ARENA's February 2022 article reported that the operating system was on track to provide 50 to 60% of the site's energy needs from renewables. The 99.99% reliability figure represents an EDL-attributed operating statement reported in an ARENA article (ARENA, 2022a). This briefing does not assert what the separate ARENA Final Report does or does not contain. The sources do not isolate the contribution of AI from the wider microgrid system. These claims are not interchangeable, nor should they collectively be presented as final, established outcomes (ARENA, 2022a; ARENA, 2022b; ARENA, 2023).

ARENA's solar forecasting project page projects cost savings of more than 20% through a reduction in Causer Pays Factors for large-scale solar farm operators. This is a forward-looking projection, not verified net commercial savings. The page does not establish net savings after implementation and operating costs, nor a basis for transferring the projection to another asset (ARENA, 2026).

AREEA's 2026 workforce research reports structured interviews with thirty-three industry leaders across resources and energy organisations. The findings reflect employer-side perspectives. Participants described hybrid technical, operational, and people-leadership roles, and task redistribution rather than simple job elimination. They raised concerns about accountability for AI-assisted decisions, surveillance, trust, privacy, and work intensification. This small, qualitative sample cannot establish sector-wide prevalence or worker sentiment. AREEA's 2024 workforce forecast concerns anticipated industry employment and project demand. It is not evidence of AI-induced employment change (AREEA, 2026; AREEA, 2024).

Taken together, these examples show specific systems being deployed and reported to deliver operational outcomes. Within the sources reviewed, however, there is no comparable portfolio dataset showing systems attempted, sustained, modified, withdrawn, or decommissioned, consistently accompanied by full lifecycle costs, and independently evaluated results. This limits what can be inferred from individual cases and makes local, decision-specific evidence essential.

4. The Governance Gap in Practice

The examples in Section 3 represent isolated commercial and public-funding deployments in the sense that each describes a single system at a single site, or a single program of work. They are useful because they show that defined systems can operate in production conditions and report measurable results. They are not a portfolio. A portfolio dataset would record every system attempted in a sector, the systems that were sustained, those that were modified or withdrawn, and the lifecycle costs and independently tested results of each. None of the sources reviewed supplies that information for Australian energy or resources operations.

The distinction matters for governance because a board that sees only successful cases will tend to overestimate the likelihood that its own proposal will succeed. Operator reporting and funding-agency reporting describe the outcomes of the projects they chose to publish. They do not, by themselves, disclose the proportion of projects that did not reach production, or the reasons. Publication bias is an analytical probability within public case literature, though it cannot be quantified from the material reviewed.

The gap also concerns the definition of production. A forecasting model that submits values through an operational interface may be described as in production, while operating at a small number of sites, under intensive vendor support, or without formal acceptance as a safety-related control. A board that does not require a definition of production - with the assets, decisions, users, hours of operation, degree of autonomy, fallback arrangements and minimum duration stated - cannot meaningfully compare one proposal with another.

Finally, the reviewed sources do not disclose enough to compare error rates, robustness under rare events, performance degradation, override frequency, or time to recovery across Australian sites. Claims of improved forecast accuracy or fault detection are incomplete unless they show whether the improvement persists under extreme weather, communications failure, sensor degradation, or operating regimes absent from the training data. The governance gap, in practice, is the difference between an encouraging case and a decision-grade evidence base.

5. The Six Board Evidence Tests

These tests are proposed governance tools for evaluating the sufficiency of evidence. They are not statutory requirements.

6. Statutory Duties and Assurance Frameworks

In the Commonwealth jurisdiction, section 19 of the Work Health and Safety Act 2011 (Cth) places the primary duty of care on a person conducting a business or undertaking (PCBU). Section 27 separately requires an officer to exercise due diligence to ensure the PCBU complies with its duties. Section 14 provides that a duty under the Act cannot be transferred. These responsibilities are distinct. The Commonwealth Act should not be assumed to govern every site. Its application, and the relevant jurisdictional text, must be checked for each operation (Commonwealth of Australia, 2011, ss 14, 19, 27).

The Act does not create an AI-specific safe harbour. Use of a vendor or an automated system should not be treated as transferring the PCBU's or officer's statutory responsibility.

Western Australia has its own Work Health and Safety Act 2020 and Work Health and Safety (Mines) Regulations 2022. Victoria operates under a separate occupational health and safety framework, including the Occupational Health and Safety Act 2004. Management should identify the legislation, regulations, and regulatory requirements applicable to each operation before relying on a general statement of duty (Western Australia, 2020; Western Australia, 2022; Victoria, 2004).

Safe Work Australia's material on AI and digital technologies is guidance. Safe Work Australia is not a WHS regulator and does not enforce jurisdictional laws. Its material identifies that automation can reduce physical exposure while changing job demands and creating psychosocial risks (Safe Work Australia, n.d.). A model Code of Practice must be approved in the relevant jurisdiction to have legal effect. Approved codes are not themselves law, though they may be admissible in court proceedings. The status of any instrument should be checked under the applicable jurisdiction's law (Safe Work Australia, n.d.).

AS ISO/IEC 42001:2023 is a voluntary management-system standard for artificial intelligence. Conformity with the standard does not confer statutory immunity, nor does certification operate as independent proof that a specific system is safe, effective, or compliant with Australian law, unless specific standards are mandated by contractual or regulatory instruments. Whether it is required in a particular context depends on applicable legal, contractual or procurement instruments. Conformity or certification should not, by itself, be treated as proof that a particular AI system is safe, effective, or legally compliant (Standards Australia, 2024).

7. Recommended Board Pathway

Use a staged decision gate proportionate to the system's operating role and consequences of failure.

The pathway above represents proportionate governance practice rather than a formal discharge of legal duties, and it does not establish that a director has satisfied section 180 of the Corporations Act 2001 (Cth). Each board must determine, on the facts of its own corporation, whether its process meets the applicable statutory standard.