With the growing use of AI systems and the increasing complexity of the legal framework relating to such use, the need for appropriate methods and tools to audit AI systems is becoming more pressing both for professionals and for regulators. The French Supervisory Authority (“CNIL”) has recently tested tools that could potentially help its auditors understand the functioning of an AI system.

Overview of the tools tested by the CNIL

The CNIL tested two different tools, IBEX and Algocate.  While IBEX aims at explaining an AI system, Algocate seeks to justify the decisions made by a AI system by checking the decision against specific standards. Both tools enable “black box” audits, meaning that they focus on the ins and outs of an AI system rather than on its internal functioning. The tools also rely on local explanatory methods, which provide an explanation for a decision related to a particular data input in the system; not on global explanatory methods which would attempt to explain all possible decisions simultaneously.

Test and conclusions

The CNIL asked some of its agents to use these tools in a theoretical scenario and consider the following questions:

  • Were the explanations provided by the tool helpful to understand the functioning of the AI system?
  • Were such explanations understandable by the participants?
  • Would these tools facilitate the work of the CNIL’s auditors?

The CNIL agents noted some challenges for each tool, in particular in relation to real-life use and the complexity of the tools.  The CNIL’s experiment also showed that some users would have preferred an explanation of the generic functioning of the system rather than local analyses. 

It therefore seems the tools will require some further improvement before they can be effectively used by regulators.  Other French public initiatives are looking into different audit models relying, for example, on global explicative methods (e.g., Pôle d’expertise de la régulation numérique’s study on methodologies for auditing content recommendation algorithms – available in French here). 

We will keep monitoring this topic moving forward, and relay any updates from the CNIL relating to auditing tools for AI systems.

Photo of Kristof Van Quathem Kristof Van Quathem

Kristof Van Quathem advises clients on data protection, data security and cybercrime matters in various sectors, and in particular in the pharmaceutical and information technology sector. Kristof has been specializing in this area for over fifteen years and covers the entire spectrum of…

Kristof Van Quathem advises clients on data protection, data security and cybercrime matters in various sectors, and in particular in the pharmaceutical and information technology sector. Kristof has been specializing in this area for over fifteen years and covers the entire spectrum of advising clients on government affairs strategies concerning the lawmaking, to compliance advice on the adopted laws regulations and guidelines, and the representation of clients in non-contentious and contentious matters before data protection authorities.

Photo of Alix Bertrand Alix Bertrand

Alix advises clients on EU data protection and technology law, with a particular focus on French privacy and data protection requirements. She regularly assists clients in relation to international data transfers, direct marketing rules as well as IT and data protection contracts. Alix…

Alix advises clients on EU data protection and technology law, with a particular focus on French privacy and data protection requirements. She regularly assists clients in relation to international data transfers, direct marketing rules as well as IT and data protection contracts. Alix is a member of the Paris and Brussels Bars.