NYC local law 144-21 tackles AI bias in employment decisions

Summary of rules within the current version of the law

Organizations that use automated employment decision tools (AEDTs) to substantially assist or replace discretionary decision-making for employment decisions (e.g., hiring, promotions, etc.) in New York City will be required to have a bias audit performed by an independent auditor. The tools include data analytics, statistical modeling, machine learning (ML), and AI that generate simplified outputs like candidate scores, classifications, or hiring recommendations.

The bias audit shall include but not be limited to testing AEDTs to assess the tools’ disparate impact on employment decisions for candidates or employees based on protected categories (e.g., sex, ethnicity, and race). In cases where individuals assessed by an AEDT are not included in the required calculations because they fall within an unknown category, organizations will need to indicate the number of these individuals. In addition, an independent auditor may exclude a category that represents less than 2% of the data being used for the bias audit from the required calculations for the impact ratio.

Organizations will need to make the date of the most recent bias audit, a summary of results, and distribution date of the AEDT prior to its use publicly available on the employment section of their websites in a clear and conspicuous manner.

Penalties for organizations that fail to comply with NYC LL 144-21 include a civil penalty of $500 for a first violation and each additional violation occurring on the same day as the first violation and a penalty between $500 and $1,500 for each subsequent violation. Each day an organization uses an AEDT in violation of NYC LL 144-21 and fails to provide any required notice to an employee or candidate shall constitute a separate violation. For further details regarding the current rules, refer to the Notice of Adoption.

Considerations regarding NYC LL 144-21

Organizations continue to evaluate the implications of NYC LL 144-21. As of the date of publication of this document, the enforcement of this law has been postponed from the originally announced date of January 1, 2023, to July 5, 2023. Key considerations or areas for clarification from stakeholders include but are not limited to:

• Types of models and applications that fall under the proposed definition of an AEDT
• Independence requirements for parties performing bias audits
• Criteria to measure disparate impact and assess bias
  
• Intersectionality of characteristics when measuring disparate impact and assessing bias and appropriate sample sizes for relevant categories
  

• Appropriate data sets to use as part of the bias audit

• Requirements of existing privacy laws and impact on availability of historical data

• Coordination between AEDT providers and employers
  

On April 6, 2023, the DCWP issued a Notice of Adoption with the final new rules and additional guidance to clarify the requirements in NYC LL 144-21.

What organizations can do now

While there are still questions related to NYC LL 144-21, there are several practices that organizations using an AEDT can consider to be proactive and prepared for the law going into effect.

• Identify any AEDTs in current use or planned for future use and determine if they substantially assist or replace discretionary decisions related to employment
• Develop an inventory of identified AEDTs to categorize and track AEDTs, including where and how AEDTs are used, and a process for tracking updates
• Identify data used by the AEDT(s) and other candidate or employee data retained by the organization and assess if the organization needs to retain additional data to perform the proposed impact calculations
• Coordinate with human resources and legal functions/departments to prepare or adapt necessary disclosures to comply with the proposed requirements