AI Fairness
Collaboration Opportunities:
Released Documents:
Standard for Fairness Assessment and Rating of Artificial Intelligence Systems
Background:
-
Artificial intelligence (AI) and Machine Learning (ML) applications are increasingly being used in all domains. Unintended biases in their predictions or outcomes are of significant concern. One crucial requirement of Responsible AI is that AI/ML Systems should be unbiased and fair. In line with the Government of India's objective to build public trust in AI/ML Systems (#AIforAll), TEC has initiated a Fairness Assessment of AI/ML Systems on a voluntary basis.
-
TEC's C&B Division invited public consultations via a letter dated February 22, 2022, conducted an interactive webinar on March 22, 2022, and hosted a public consultation meeting on September 1, 2022, to frame procedures for assessing fairness for different types of AI/ML Systems and for issuing fairness ratings/certifications as benchmarks.
-
Following the public consultations, TEC established a Working Group comprising members from the industry, academia, researchers, subject experts from government departments, and others. This group was tasked with preparing the initial draft of the proposed Standard for assessing the fairness of AI Systems.
-
Based on extensive stakeholder consultations and inputs from domain experts, TEC unveiled a New Standard (No. TEC 57050:2023) for "Fairness Assessment and Rating of Artificial Intelligence Systems" on July 7, 2023. This Standard outlines detailed procedures for assessing and rating artificial intelligence systems for fairness.
The next steps, collaboration opportunities:
- Developing tools for fairness evaluation - Libraries, software development kits (SDKs),
- Assuming the role of fairness auditors - labs of academic institutes, R&D organisations designated by TEC
- Partnering to extend the standard to other data modalities like text, image, speech, etc. and other kinds of ML
- Collaborating with TEC for joint research, co-supervision of Ph.D. students
Other documents and links:
- TRAI releases Recommendations on Leveraging Artificial Intelligence and Big Data in Telecommunication Sector, 20-07-2023
- TEC OM dated 14-10-2022, 28-09-2022 & 19-09-2022 regarding constitution of a Working Group for preparing the initial draft of the proposed Standard for fairness assessment and rating of AI systems.
- Record of Discussions- Consultation Meeting held on 01-09-2022 on Framework for Fairness Certification of Artificial Intelligence / Machine Learning Systems.
- Video recording of the TEC Webinar held on 22-03-2022 on Framework for Fairness Certification of Artificial Intelligence Systems.
- Brochure of the TEC Webinar held on 22-03-2022 on Framework for Fairness Certification of Artificial Intelligence Systems.
- TEC letter dated 22-02-2022 inviting inputs for creating Framework for Fairness Assessment of Artificial Intelligence (AI)/ Machine Learning (ML) Systems #AIforAll.
- News coverage of TEC's initiative regarding Fairness Assessment of AI/ML.
Contact Us:
DDG (C&B) TEC: avinash.70@gov.in
Director (C&B) TEC: dircb2.tec-dot@gov.in
References:
1] Agarwal, A., Agarwal, H. & Agarwal, N. Fairness Score and process standardization: framework for fairness certification in artificial intelligence systems. AI Ethics 3, 267–279 (2023).
[2] Agarwal, A., Agarwal, H. A seven-layer model with checklists for standardising fairness assessment throughout the AI lifecycle. AI Ethics (2023).