Section 1 Short title
This Act may be cited as the “Validation and Evaluation for Trustworthy (VET) Artificial Intelligence Act” or the “VET Artificial Intelligence Act”.
Validation and Evaluation for Trustworthy (VET) Artificial Intelligence Act
A BILL
To require the Director of the National Institute of Standards and Technology to develop voluntary guidelines and specifications for internal and external assurances of artificial intelligence systems, and for other purposes.
Sec. 2 Purposes
The purposes of this Act are—
to develop consensus-driven, evidence-based voluntary technical guidelines and specifications for internal and external assurances through the testing, evaluation, validation, and verification of artificial intelligence systems, as appropriate based on the intended application, use-case, and risk profile of the artificial intelligence system;
to use meaningful assurance to supplement methodologies used to build trust in artificial intelligence systems, increase adoption of artificial intelligence systems, and provide for accountability and governance of artificial intelligence systems; and
to further the goals of the Artificial Intelligence Risk Management Framework, including any successor framework, published by the National Institute of Standards and Technology and the Artificial Intelligence Safety Institute pursuant to section 22A(c) of the National Institute of Standards and Technology Act (15 U.S.C. 278h–1(c)).
Sec. 3 Definitions
In this Act:
Artificial intelligence system— The term artificial intelligence system means a machine-based system that, for explicit or implicit objectives, infers, from the input the system receives, how to generate outputs, such as predictions, content, recommendations, or decisions, that can influence physical or virtual environments.
Deployer— The term deployer means an entity that operates an artificial intelligence system for internal use or for use by a third party.
Developer— The term developer—
means an entity that builds, designs, codes, produces, trains, or owns an artificial intelligence system for internal use or for use by a third party; and
does not include an entity that is solely a deployer of the artificial intelligence system.
Director— The term Director means the Director of the National Institute of Standards and Technology.
External artificial intelligence assurance— The term external artificial intelligence assurance means an independent and impartial evaluation of an artificial intelligence system conducted by a nonaffiliated third party in accordance with the voluntary assurance technical guidelines and specifications described in section 4 or consensus-driven voluntary standards, for the purpose of—
verifying claims with respect to the functionality and testing of the artificial intelligence system, including verifying whether it is fit for its intended purpose; or
identifying any significant error or inconsistency in the testing, risk management processes, or internal governance, any substantial vulnerability, or any negative societal impact of the artificial intelligence system.
Internal artificial intelligence assurance— The term internal artificial intelligence assurance means an independent evaluation of an artificial intelligence system conducted by the party being evaluated with an internal reporting structure that encourages impartial evaluations and prevents conflicts of interest, for the purpose of—
verifying claims with respect to the functionality and testing of the artificial intelligence system, including verifying whether it is fit for its intended purpose; or
identifying any significant error or inconsistency in the testing, risk management process, or internal governance or any substantial vulnerability of the artificial intelligence system.
Nonaffiliated third party— The term nonaffiliated third party with respect to the evaluation of an artificial intelligence system, means a person who—
is not related by common ownership or affiliated by common corporate control with the developer or deployer of the artificial intelligence system;
can demonstrate financial independence from the developer or deployer of the artificial intelligence system;
does not employ any individual, who is also employed by the developer or deployer of the artificial intelligence system; and
is a qualified evaluator of artificial intelligence systems, with—
demonstrated expertise in relevant technical domains, including—
data privacy and security principles; and
risk management practices in artificial intelligence systems; and
familiarity with the relevant details regarding the type of artificial intelligence system being evaluated.
Secretary— The term Secretary means the Secretary of Commerce.
Sec. 4 Voluntary assurance technical guidelines and specifications for artificial intelligence systems
Voluntary technical guidelines and specifications for assurance— Not later than 1 year after the date of the enactment of this Act, the Director, in collaboration with public and private sector organizations, including the National Science Foundation and the Department of Energy, shall develop and, not less frequently than every 2 years, shall review and update as the Director considers appropriate, a set of voluntary technical guidelines and specifications for internal artificial intelligence assurance and external artificial intelligence assurance.
Contents— The technical guidelines and specifications required by subsection (a) shall—
identify consensus-driven, voluntary standards for internal artificial intelligence assurance and external artificial intelligence assurance that address—
safeguards for consumer privacy;
methods to assess and mitigate harms to individuals by artificial intelligence systems;
dataset quality;
documentation, disclosure, and provenance communications to external parties; and
governance and process controls;
provide technical guidelines, best practices, methodologies, procedures, and processes, as appropriate, for internal artificial intelligence assurance and external artificial intelligence assurance that effectively address the elements listed in paragraph (1);
establish common definitions and characterizations for testing, evaluating, verifying, and validating methods for internal artificial intelligence assurance and external artificial intelligence assurance;
recommend criteria or approaches for a developer or deployer to determine the frequency and circumstances under which internal artificial intelligence assurance and external artificial intelligence assurance activities should be conducted, accounting for the relevant risk and use-case profile of the artificial intelligence system, and any additional circumstance under which an assurance should be conducted;
recommend criteria or approaches for a developer or deployer to determine the scope of internal artificial intelligence assurance and external artificial intelligence assurance conducted through testing and evaluating, accounting for the relevant risk and use-case profile of the artificial intelligence system, including the minimum information or technical resources that should be provided to the party conducting the assurance to enable assurance activities;
provide guidance for the manner in which a developer or deployer may disclose, as appropriate, the results of an internal or external assurance or carry out corrective actions with respect to an artificial intelligence system following the completion of an internal or external assurance of such system, and guidance on the manner in which a developer or deployer may properly document any corrective action taken;
align with the voluntary consensus standards, including international standards, identified pursuant to paragraph (1) to the fullest extent possible;
incorporate the relevant voluntary consensus standards identified pursuant to paragraph (1) and industry best practices to the fullest extent possible;
not prescribe or otherwise require—
the use of any specific solution; or
the use of any specific information or any communications technology product or service; and
recommend methods to protect the confidentiality of sensitive information, including personal data and proprietary knowledge of an artificial intelligence system, that may be obtained during the assurance process.
Stakeholder outreach— In developing the voluntary technical guidelines and specifications required by subsection (a), the Director shall—
solicit public comment on at least 1 draft of the technical guidelines and specifications, and provide a reasonable period of not less than 30 days for the submission of comments by interested stakeholders;
make each complete draft of the voluntary technical guidelines and specifications developed under subsection (a) available to the public on the website of the National Institute of Standards and Technology; and
convene workshops, roundtables, and other public forums, as the Director considers appropriate, to consult with relevant stakeholders in industry, academia, civil society, consumer advocacy, workforce development organizations, labor organizations, conformance assessment bodies, and any other sector the Director considers appropriate, on the development of the voluntary technical guidelines and specifications.
Publication— The Director shall publish the voluntary technical guidelines and specifications required by subsection (a) as a standalone framework or document available to the public on the website of the National Institute of Standards and Technology.
Sec. 5 Qualifications advisory committee
Advisory committee— Not later than 90 days after the date on which the Director publishes the voluntary technical guidelines and specifications required under section 4(a), the Secretary shall establish the Artificial Intelligence Assurance Qualifications Advisory Committee (referred to in this section as the “Advisory Committee”).
Membership— The Secretary shall appoint to the Advisory Committee not more than 20 individuals with expertise relating to artificial intelligence systems, including at least 1 representative from each of the following:
Institutions of higher education.
Organizations developing artificial intelligence systems.
Organizations deploying artificial intelligence systems.
Organizations assessing artificial intelligence systems.
Consumers or consumer advocacy groups.
Public health organizations.
Public safety organizations.
Civil rights organizations.
Professional accreditation organizations.
Workforce development organizations.
Labor organizations.
Nonprofit assurance professional organizations.
Duties— The Advisory Committee shall—
review and assess case studies from entities that provide licensure, certification, or accreditation to independent organizations with a primary mission of verifying compliance with applicable statutes, regulations, standards, or guidelines; and
determine the applicability of the case studies reviewed and assessed under paragraph (1) to the development, maintenance, and use of artificial intelligence systems for the purpose of developing recommendations under subsection (d).
Recommendations— Not later than 1 year after the date on which the Secretary establishes the Advisory Committee under this section, the Advisory Committee shall submit to the Secretary and Congress and make publicly available a report that includes recommendations for the Secretary to consider regarding—
the qualifications, expertise, professional licensing, independence, and accountability that a party conducting an assurance of an artificial intelligence system should have, including with respect to the type of artificial intelligence system under evaluation and the internal and external assurance processes; and
whether accreditation for internal artificial intelligence assurance and external artificial intelligence assurance can be met through a combination of existing licensure, certification, or accreditation programs.
Termination— The Advisory Committee shall terminate not later than 1 year after the date on which the Advisory Committee submits the recommendations required under subsection (d).
Sec. 6 Study and report on entities that conduct assurances of artificial intelligence systems
Study— Not later than 90 days after the date on which the Director publishes the voluntary technical guidelines and specifications required under section 4(a), the Secretary shall commence a study to evaluate the capabilities of the sector of entities that conduct internal artificial intelligence assurances and external artificial intelligence assurances.
Considerations— In carrying out the study required by subsection (a), the Secretary shall—
assess the capabilities of the sector of entities described in subsection (a) with respect to personnel, technical tools, evaluation methods, computing infrastructure, and physical infrastructure and whether such capabilities are adequate for providing internal artificial intelligence assurances or external artificial intelligence assurances that comport with the voluntary technical guidelines and specifications required under section 4(a);
review the features, best practices, and safeguards employed by such entities to maintain the integrity of confidential or proprietary information of a developer or deployer during an internal artificial intelligence assurance or an external artificial intelligence assurance;
assess the market demand for internal artificial intelligence assurances and external artificial intelligence assurances and the availability of such assurers; and
assess the feasibility of leveraging an existing facility accredited by the Director under the National Voluntary Laboratory Accreditation Program established under part 285 of title 15, Code of Federal Regulations, to conduct external assurances of artificial intelligence systems.
Report— Not later than 1 year after the date on which the Secretary commences the study required by subsection (a), the Secretary shall submit to the appropriate committees of Congress and the head of any Federal agency that the Secretary considers relevant, a report that contains the results of the study required by subsection (a), including—
recommendations for improving the capabilities and the availability of the entities assessed in the study;
descriptions of the features, best practices, and safeguards of the entities studied and the effectiveness of such features, practices, or safeguards at implementing the voluntary technical guidelines and specifications required under section 4(a) and at maintaining the integrity of confidential and proprietary information, as described under subsection (b)(2); and
any conclusions drawn from the assessment of the facilities described in subsection (b)(4).
Appropriate committees of Congress defined— In this section, the term the appropriate committees of Congress means—
the Committee of Commerce, Science, and Transportation of the Senate; and
the Committee on Science, Space, and Technology of the House of Representatives.