Frontier AI Safety Oversight

Summary#

S-10701, called the “TERMINATOR Act” in the bill text, would amend New York’s General Business Law, including provisions added by a 2026 law, and add new safety rules for frontier AI models. Its stated goal is to strengthen safety evaluation, monitoring, reporting, security, employee protections and enforcement; those rules would not guarantee that models are safe. The bill is not law, and its passage is uncertain.

  • Large frontier developers would have to arrange independent safety evaluations before deploying models in New York, and pay the evaluators’ reasonable costs. Evaluations would test specified capabilities and safeguards; the office could add topics by regulation.
  • Developers would have to reevaluate models before certain material changes, monitor models they still control, report significant safety incidents, and keep specified safety and security records for at least five years.
  • Developers providing privileged access or transferring model weights would face added security and risk-assessment duties. Where there is a reasonable basis to find a material and unreasonable risk of catastrophic harm, developers would have to use safeguards to mitigate it.
  • The office could require further safeguards or temporarily restrict certain activities under stated conditions. The attorney general could seek a court order to address an imminent, unreasonable risk.
  • The bill would protect employees and contractors who make specified safety disclosures from retaliation. The attorney general could enforce those protections; the bill does not create a private right to sue.
  • Large frontier developers would be assessed pro rata to cover the office’s operating costs, including evaluation oversight, technical work and enforcement.

What it means for you#

  • AI developers: If covered, you would have to pay for independent evaluations, provide evaluators necessary access, report certain incidents, monitor models within your control, and keep specified records. The bill does not require evaluators to certify a model as safe.
  • Developers and third parties with privileged access: Developers would need a written security program. If access is provided to a third party, the developer must require appropriate security measures by contract. Developers must suspend or change access when they reasonably determine it creates a material and unreasonable risk of catastrophic harm.
  • Employees and contractors: You would be protected from retaliation for specified safety disclosures to the developer, evaluator, government office, attorney general or another appropriate government entity. The office would have to provide a confidential reporting mechanism. Enforcement would be through the attorney general, not a private lawsuit under this bill.
  • AI users and the general public: The bill does not ban frontier models or require general logging of ordinary user prompts and communications. Monitoring must minimize collection and retention of personal information and unrelated user communications where reasonably feasible.
  • Taxpayers: The bill directs that developers, not the public generally, be assessed for the covered operating costs. The materials do not say whether those assessments would cover every cost in practice.

Money#

The bill requires developers to pay evaluation costs and be assessed for administration, but the available material gives no dollar estimates.

  • Developer assessments may cover direct and indirect costs, including staff, technical experts, evaluator oversight, investigations, rulemaking, information security and enforcement.
  • The bill allows civil penalties of up to 0.5% of annual gross revenue for specified knowing conduct, and up to 1% for specified willful conduct. These are maximums, not automatic fines.

What is unclear#

  • The supplied material does not include the 2026 law that established the existing provisions. I could not verify the full current law, including definitions, the office’s identity, or how these new rules would fit with existing duties.
  • The bill relies on terms such as “large frontier developer” and “frontier model”; their definitions are not included in the supplied text. Its definition of “catastrophic harm” also refers to a part of existing law not supplied.
  • Regulations would set evaluator qualifications, monitoring standards and cybersecurity requirements. Their details, and how many evaluations or reviews would be needed, are not yet known.
  • The bill sets a start date of one year after it becomes law, but ties that date to when the 2026 law takes effect. The supplied material does not establish when either law would take effect.
  • The bill does not give cost estimates or specify how the office would decide whether developer assessments fully cover its costs.

Case for#

  • A possible argument for the bill is that independent testing could provide a check on developers’ own safety assessments before deployment.
  • Required reevaluations, monitoring and incident reports could help identify new risks after deployment or after a material change.
  • The bill limits restrictions to identified risks and requires written findings; in most cases, the developer would get notice and a chance to respond.
  • Employee protections could make it safer to report concealed incidents or interference with evaluations.

Case against#

  • The bill would give the office broad roles in setting technical rules, reviewing evaluations and restricting activities. Key standards would depend on later regulations.
  • Independent evaluations, monitoring, recordkeeping and security requirements could impose substantial costs and work on covered developers; the bill provides no estimates.
  • The bill does not set out detailed procedures for every evaluation or office review, such as timelines for some decisions or a general appeals process.
  • Temporary restrictions could take effect without prior notice when the office finds that notice would increase imminent risk. The bill provides later review opportunities, but a restriction could be renewed.