Context
Artificial intelligence (AI) is a rapidly evolving field with immense potential. It holds the promise to transform societies, stimulate economic growth, and lead to all manner of discoveries. At the same time, many governments and individuals are concerned about the potential adverse effects of AI if left unchecked, and some are saying governments must regulate now before it’s too late.
When it comes to regulation, Europe has been ahead of the curve with its proposed Artificial Intelligence Act (AI Act). The European Commission introduced the proposal for the AI Act in April 2021. The Act aims to balance the benefits of AI with the imperative to ensure safety, security, and the protection of fundamental rights. It proposes a technology-neutral definition of AI systems and adopts a risk-based approach, categorising AI systems into four levels of risk. Those levels are described on the European Parliament website as follows:
- Unacceptable risk AI: Harmful uses of AI that contravene EU values (such as social scoring by governments) will be banned because of the unacceptable risk they create.
- High-risk AI: A number of AI systems (listed in an Annex) that are creating adverse impact on people’s safety or their fundamental rights are considered to be high-risk. In order to ensure trust and consistent high level of protection of safety and fundamental rights, a range of mandatory requirements (including a conformity assessment) would apply to all high-risks systems.
- Limited risk AI: Some AI systems will be subject to a limited set of obligations (e.g. transparency).
- Minimal risk AI: All other AI systems can be developed and used in the EU without additional legal obligations than existing legislation.
The proposed Act is currently under deliberation by the European Parliament and the Council of the European Union.
Whether New Zealand will follow suit with its own AI legislation remains to be seen.
Draft standard contractual clauses for the procurement of AI by public organisations
Running in parallel, work is underway with the support of the European Commission to develop EU standard contractual clauses for the procurement of AI by public organisations. As at the date of this post, the latest draft is 0.9.1 and can be found on the European Commission’s Public Buyers Community website. Two versions are available: a full version for high-risk AI systems within the meaning of the draft AI Act, and a light version for non-high risk AI systems. The clauses “only contain provisions specific to AI Systems and on matters covered by the proposed AI Act”.
While in draft and therefore not an official EU document, the proposal for standard contractual clauses is particularly helpful for New Zealand public sector agencies who may, in time, come to procure and contract for AI systems. The clauses cover a lot of ground and are helpful in triggering thoughts as to what a set of New Zealand-oriented clauses may need to cover.
The proposed clauses in the full version address the following topics:
Section A – Definitions
Article 1: Definitions
Section B – Essential requirements in relation to the AI-system
Article 2: Risk management system
Article 3: Data and data governance
Article 4: Technical documentation and instructions for use
Article 5: Record-keeping
Article 6: Transparency
Article 7: Explainability
Article 8: Human oversight
Article 9: Accuracy, robustness and cybersecurity
Section C – Obligations of the Supplier in relation to the AI System
Article 10: Compliance with Section B
Article 11: Quality management system
Article 12: Conformity assessment
Article 13: Automatically generated logs
Article 14: Corrective actions
Article 15: Post-market monitoring
Section D – Rights to use the Data Sets
Article 16: Rights to the Public Organisation Data Sets
Article 17: Rights to Supplier Data Sets and Third Party Data Sets
Article 18: Hand over of the Data Sets
Article 19: Indemnification
Section E – AI register and audit
Article 20: AI register
Article 21: Compliance and audit
Section F – Costs
Article 22: Costs
The clauses are well worth a read by any public sector organisation contemplating the procurement of an AI system, as they’re a valuable resource for understanding key issues at stake and potential strategies for addressing them.