Committee on Technology

Good morning and welcome to today's New York City Council hearing for the Committee on Technology. At this time, I would like to remind everyone to silence all electronic devices. Also, at this point going forward, no one is supposed to approach the dais. Chair, we're ready to begin. Thank you.

Good morning, everyone. I'm New York City Council Speaker Julie Menin. I want to welcome everyone to today's hearing. I want to thank my colleague, the Technology Chair, Councilmember Carmen De La Rosa. We look forward to having a very fulsome hearing, and this is a sequel to our initial hearing on AI. Today, the New York City Council continues a crucial conversation that continues from last week's Committee of the Whole with the world's largest AI companies.

We were the first legislative body in the country that heard from the leading companies on their models that have captured as much attention as they have concern across the globe. As we heard from both experts and the public, there is good reason to remain apprehensive about some of the safety risks around artificial intelligence, as well as the companies behind these AI models.

The answers, quite frankly, that we received from last week's hearing only reinforce the necessity for responsible safeguards to prevent harm. And the idea that the companies would not— would self-regulate is not a concept that we support at all. At the same time, we want to ensure that we are not stifling innovation. New York City is rapidly growing as not only a tech capital, capital, but also the AI capital of the world with over 400,000 jobs, more than the city of San Francisco, I might add.

And we understand and we want to support keeping those jobs here in New York, supporting cutting-edge research, supporting startups locating right here in our great city. So we know that no matter how large or how small these various AI companies are, We know that those companies are vital to our city in terms of cutting-edge research, medical breakthroughs, scientific discoveries, and boosting economic growth.

But we cannot wait for these companies to act responsibly when we see that some of these models are able to hack into other companies and also into government websites as well. As we work to ensure that there are responsible, suitable safeguards in place following alarming public reports on AI. We want to welcome SpaceX to this hearing. It is not only New Yorkers who want to hear your testimony today, but people across the world.

So we thank them for being here under oath to answer our questions about this evolving technology. Like I said at the hearing last week, these companies are developing technology that who can reshape so many aspects of our lives. And yet they are being led oftentimes by individuals who are publicly warning about the risks with their models. So we deserve to have those risks, those safety concerns answered, as well as a discussion about what are the best legislative solutions to address those risks.

This city council has proposed solutions to address, for example, emergency preparedness. We have bills to require third-party independent validation that is free from conflict of interest. We have a bill, first in the nation, to incentivize whistleblowers to come forward, as well as a bill to devise a plan for the scenario in which, in case AI hacks into city systems, as well as other bills as well.

I want to end by saying that at our hearing last week, we showed that New York city has demonstrated to the world that cities can and should take the lead on AI safety. Our federal government is simply failing to act. Of course, we would like to see regulation at the federal level, and it is our fervent hope that by putting forward these legislative ideas, that it will ultimately lead to the federal government coming up with national safety standards and regulations of AI.

Just as I said last week, we do not leave it to the pharmaceutical industry to self-regulate. We have the FDA. We do not leave it to the airline industry to self-regulate. We have the FAA and an independent safety board when there are safety concerns. We have the SEC that regulates the financial industry. With AI, what we are looking for is responsible safeguards to put in place.

New York City has and always will be a leader in addressing the key issues that affect New Yorkers, and AI is simply no different. I look forward to today's conversation. I want to thank our team, our staff, for helping to organize this important hearing. And now I want to turn it over to Chair De La Rosa for a statement.

Thank you. Thank you, Speaker Menin. Good morning, everyone. I'm Councilmember Carmen De La Rosa, and I am the chair of the Committee on Technology. Welcome to our hearing on continuing discussions about the risks posed by artificial intelligence. Last week, in this very same room, in a meeting of the Committee of the Whole, we heard testimony from nearly 100 witnesses, including representatives from OpenAI, Meta, Anthropic, Google, their former employees, tech technical experts, academics, elected officials, and members of the public.

Despite their different points of view, their message was more or less consistent. The pace and progress of frontier AI models does pose a risk. Some called for frontier developers to slow down. Others focused on alignment and control mechanisms, warnings of superintelligence, mass surveillance, and agentic policing, as well as political bias and discriminatory tendencies. Several witnesses raised concerns about the loss of model control, and others warned about the potential loss of jobs.

However, much of the testimony returned to a common set of recommendations: transparency from AI developers, credible safety measures, third-party evaluators, and protection for personal data. Today, we plan to continue this conversation with another major AI company, SpaceX AI, We want to reap the benefits of AI so long as these benefits come with responsible development and do not cost us our safety, our dignity, and our civil rights.

AI has tremendous potential and can be a powerful tool for efficiency and innovation in science, medicine, and productivity. At the same time, it introduces important risks including bias, discrimination, job displacement, censorship, digital forgeries, deepfakes, and concerns that may become even more serious as AI agents become more capable. We should make sure AI tools work as intended, that we understand how they work, how they were trained, where they fail, and ensure that those affected have an opportunity to contest decisions made by machines.

A framework where the very same people who build AI tools and have competing motives are also expected to regulate themselves is wholly insufficient. Intervention is necessary from federal, state, and municipal governments to protect our rights. Once again, I look forward to hearing more from industry experts about what solutions are possible to ensure New Yorkers can fully benefit from technological advancement without compromising civil rights, safety, and privacy.

I'd now like to thank the Technology Committee staff, Legislative Counsel Irene Byofsky and Policy Analyst Eric Brown, as well as my Chief of Staff James Burke, Legislative Director Hiba Imad, Comms Director Frey Familia, and Speaker Menin's entire team for their continued work on this important matter. I'd also like to recognize some of the councilmembers that have joined us.

Councilmember Maloney. And that's it for now. More to come. I now turn it back to Speaker Menin and the committee counsel.

Thank you so much, Chair De La Rosa. I'm now going to pass it over to counsel to do the affirmation. Ms. Castilla-Madrid, you may answer yes after the question. Do you affirm to tell the truth, the whole truth, and nothing but the truth in your testimony before this committee? Yes. Thank you. You may begin your testimony.

Madam Speaker, Chair De La Rosa, and members of the Committee on Technology, thank you for the opportunity to participate today. My name is Gabriela Castillo Madrid, and I am part of the government affairs team at SpaceX. My team's responsibility is to maintain a close relationship between the teams building our advanced technologies, including Frontier SI models, and the communities that use and benefit from them.

SpaceX has grown from a startup nearly 25 years ago to a company that launches more rockets and orbits more satellites every year than any other nation or company. Developing rapidly reusable rockets has enabled SpaceX to dramatically lower the cost of launch and make new NASA missions and science possible, including regular and reliable crew and cargo missions to the International Space Station and the upcoming Artemis mission to the Moon.

Our Starlink satellites now make possible reliable broadband connectivity across over 160 countries, including to locations where high-speed internet has never been available before. SpaceX's mission is to make humanity multiplanetary. Earlier this year, SpaceX acquired xAI, blending SpaceX's deep technical expertise building large, complex infrastructure and complex space systems with the development of superintelligence and frontier SI models. Just as we have done with space launch and human spaceflight, SpaceX SI puts safety first.

As a mother, I know the risks New Yorkers worry about are often those close to home: protecting our children, preventing harmful content, and stopping misuse of everyday SI tools. And I can assure you that safety is core to how we build Grok, from pre-training through deployment. Our approach starts with a simple idea: a superintelligent system built to seek the truth and understand the universe as it actually is will be more safe, reliable, and useful to the people who rely on it.

Our safety work is how we make sure that holds up in practice by focusing on control, prevention, mitigation, analysis, and correction. Concretely, that effort begins well before a model is released for public use. We train our models to refuse assistance with dangerous activity, evaluate it for risks including misuse of our terms of service, attempts to get around safeguards, and loss of control.

We monitor systems after deployment, and we do not release models we believe are insufficiently robust against misuse by bad actors or follow meaningful human control. These practices are outlined in our publicly available Frontier Artificial Intelligence Framework along with each new model card. which describes how we assess and reduce these risks across the model's lifecycle. We also perform a full risk assessment of our practices and controls at least once per year.

We believe transparency is fundamental to creating a safer ecosystem. When something goes wrong, we investigate it, work to fix it, and report it to the authorities when required. Used well, SI can help improve the lives of the people of New York, By speeding up research, making public information easier to find and use, and helping city agencies and residents with complex tasks.

Those benefits only matter if the systems remain safe and accountable. This is why we continue to invest in evaluation, monitoring, and human control, and why we want to work with policymakers on practical, evidence-based protections. I look forward to answering your questions today and continuing the conversation. Thank you.

Thank you very much for your testimony. So we have a number of questions. On September 12th, Dario Mode authored an article entitled We Must Pace the Frontier, stating that within 6 to 12 months, a swarm of AI agents, quote, could be capable of taking over the entire internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage, end quote.

Was he correct here? I wouldn't— I'm not familiar with that statement in particular. What I will say is safety is core to SpaceX AI, and that goes from pre-training through release. So people stay in control of our systems at every stage. We do not release models we do not believe are safe and that we cannot keep under meaningful human control.

And if a model does not meet that bar, we hold it back and strengthen the model and safeguards. We train models to ensure they don't assist with dangerous activity and evaluate them for misuse. So then why did Elon Musk state on X in a post in response to those comments, Dario is right? I believe we're optimistic about SI.

We obviously take the risks seriously. Elon has been one of the earliest and most prominent voices in warning about SI risk. and calling for oversight. But we are extremely positive about what can be achieved. We believe that superintelligence can help people solve problems that were previously beyond reach, accelerating scientific discovery and expanding knowledge and creating new opportunities for human progress.

So you've just mentioned that the company will not release any models that are not, quote unquote, safe. So will you Agree then that the company will not release any model that failed an internal or third-party validated test? Yeah, the public can take great confidence in the testing that occurs before models are released and the safeguard stacks that surround model deployments.

As models get more capable, we often find gaps that need stronger protections, and we also tighten those before the model goes out. So to that end, Axios reported on October 9th that major AI companies were, quote, privately gaming out scenarios for a public and political revolt after a catastrophic AI event, end quote. Has your company ever conducted an internal assessment or worst-case outcome causing substantial financial, cybersecurity, or physical harm?

I'm not familiar with any specifics, but I will say that we take catastrophic risk very seriously. I just want to interrupt you when you say you're not familiar with any specifics. I mean, the question is, has Axios had this reporting saying that many of the frontier companies were doing— were gaming these tests out? Is that something that SpaceX AI has done or not?

I cannot speak to a specific test. What I can say is that we do obviously test our systems and make sure there's continuous evaluation and human oversight rather than accepting any unacceptable Is the company aware of any safety or cybersecurity incident or any evidence of such an incident that has not yet been publicly reported? I'm not aware of any incidents that have not been reported.

So, I mean, I just find this answer not sufficient. I mean, you're not aware. Is the company— I'm going to rephrase the question. Is the company aware? Safety is the core of everything we do. And like I mentioned, we wouldn't release a model that hasn't gone through thorough review and making sure that we can keep it under meaningful human control.

And like I said, if the model does not meet that bar, we hold it back and strengthen the model and the safeguards around it. So when you say that you have— the company doesn't release any model that is not safe.

In August 2025, the company launched Grok Imagine with a quote unquote spicy mode for explicit content. In late December 2025, the company integrated Grok's image editing features into the platform, allowing users to alter photos of real people. Within days, Grok generated millions of sexualized images of women and children. Approximately 23,000 images appeared to depict young children. So how do you then, when you say you're not releasing a model that is safe, do you think that that is acceptable conduct?

To be clear, that's not acceptable conduct, and we're all on the same side of this. Grok has strict controls in place to prevent the sexualization of children or real people. We've always had prohibited CSAM and NCII. When bad actors fail, go around our safeties, we make sure that we catch them and report them to authorities. We actually have a great partnership with the National Center for Missing and Exploited Children.

And just this year, we have made over 100,000 reports that have resulted in almost 300 arrests. So today, can a user get Grok, the Grok app, or Grok Imagine to show a real person nude, in their underwear, or in a sexualized pose? Grok has strict controls in place to prevent the sexualization of children or real people. Yeah, I'm gonna completely push back on it.

I mean, just this morning, myself and my team, we put our own name, my own name, into Grok, and there's absolutely, like, a sexualized image of myself. So if it can happen to me, it can happen to any woman. So I completely discount what you're saying, that Grok has solved this problem. I mean, it's completely unacceptable when these kind of sexualized images are readily available.

To be clear, no one should have sexualized images made of them without their consent, and our policies prohibit that. Any specific cases I can't speak of, or— but I will say, like I said before, we have a very bright line, and we— Grok cannot be used to exploit minors, commit crimes, or generate illegal content. But so then how is it that within a second I could find a sexualized image of myself on using Grok?

Madam Speaker, I can't speak to that specific instance. What I can say is that we do have strict controls, and I'm happy to get our safety team to look into that specific instance. And if you had to report— you mentioned 100,000 instances— doesn't that mean then, by that very definition, that the technology allows it? Our reports are meant to catch bad actors.

Unfortunately, there's always going to be bad actors and folks trying to get around safeguards. And our, our work is to catch them and get them to the right authorities. How many reports, complaints, takedown requests, or demands from depicted individuals has the company received concerning Grok-generated sexualized images? You mentioned the 100,000. Does that encompass all of the complaints that people have filed on this issue?

That over 100,000 reports are reports specifically sent to the National Center for Missing and Exploited Children. In the case that you are explaining, we— the reports were caught and actions were taken. And we even took the— took 3 federal lawsuits against the users that were already facing criminal charges for their abuse of Grok in creating illegal content.

But then the fact that within seconds I could find this image of myself, like, what type of safeguards is Grok going to do then moving forward? Well, like I said, we'll make sure that we're strengthening our systems. And I, I cannot speak to that particular case, but we do have strict controls in place to prevent the sexualization of children or real people.

In July 2025, Grok's chatbot published a series of antisemitic posts, including one in which it called itself Mecha Hitler. What steps has SpaceX AI taken to prevent these instances from happening? We, we take steps to strengthen all of our safeties, and we train models to ensure they don't assist with dangerous activity, evaluate them for misuse. which was the case, and attempts to get around safeguards and loss of control.

And we monitor after deployment as well. We understand that it has to be an iterative process. As models evolve, our, our safety has evolved with it as well. How many SpaceX AI employees are principally dedicated to identifying, evaluating, and mitigating safety, security and cybersecurity risks associated with your frontier AI models? We have a robust safety team, and safety is— I'm sorry, how many?

I'm sorry, could you repeat that? We have a robust safety team, and safety is embedded into our— But how many? How many employees? I don't have the number with me right now, but what I can say is that safety is at the core of everything we do. And it's core to not just our engineers and our safety, but everybody in the company is bought into the importance of safety.

Like I mentioned earlier, we have— we're launching rockets into space. And obviously to get to that place, we need to make sure that safety is at the forefront of everything we do. This seems like a very rudimentary question. How many people are dedicated to the on the safety team relative to the company as a whole? This seems like a very basic question.

I understand your question. The way we see it is everybody is in charge of safety, and, and we make sure that if anybody sees something, they say it and report it quick. I think it would assure the public far more if we got a straightforward answer on actually how many people at the company are dedicated to evaluating the safety models?

This is a very basic question, and I'm going to ask that by the end of the testimony you provide that answer to us. Madam Speaker, we'll, we'll get on that. Thank you.

Uh, does SpaceX AI use financially independent evaluators, evaluators that do not have any direct or indirect financial ties to the company, before testing or deploying a potentially dangerous AI model? As part of our— the White House Accord, we agreed on certain agreements. Company responsibility is one of them. Internal controls, internal oversight team, and independent external evaluators is one of them as well, as well as board-level oversight.

So I guess yes or no, are there independent third-party validators who are validating who are free from any conflict of interest, who are validating the new models? We're working on partnerships with independent external auditors or evaluators to assess model capabilities, controls, and safeguards. And are you going to be disclosing who those auditors are? That's not up to me, but I'll make sure that I'll get an answer for you on that.

Again, I think, you know, the idea that there is self-regulation is not something that is making people feel comfortable at all. And so as a most rudimentary measure, to have independent third-party validators who are free from conflict of interest, which is actually what one of our bills exactly does, is something that we believe you should be able to answer.

Last week, rogue Anthropic AI agents filed a false police report in Philadelphia. Another agent attempted to file 20 false visa applications. Thankfully, the incidents did not go beyond that, but they raise the prospect of swarms of rogue agents filing thousands of false reports with government and businesses alike. If that harm occurs, who bears the legal responsibility? Madam Speaker, to be clear, we're not self-regulated.

SI is not a self-regulated industry. We operate within a framework of existing laws, and we would never use SI as an excuse for, for something or as a legal defense. But don't you agree there's no national— there's no federal regulation of AI right now? There's no law on the books that is regulating it. There's no agency. that is independent federal agency that is in charge of validating the safety models.

None of that exists. That is why we're open to working with policymakers to make sure that we find common sense, technically feasible solutions. But we do abide by regulatory requirements and government oversight. And what's illegal without superintelligence is also illegal with it. But going back to the question I asked earlier of the example of the rogue Anthropic agents filing the false reports, who bears legal responsibility?

I'm not familiar with that specific case. We're responsible if you're following the law when it comes to SpaceX AI, and existing laws apply to AI just as it applies to anything else. So if SpaceX AI causes Either material injury, physical damage, God forbid a death, something cataclysmic, then the company bears responsibility? We're responsible for following the law.

Does SpaceX AI have catastrophic insurance in the event of one of these catastrophes happening? I'm not aware of the specifics of our insurance policies and what we have and we don't. Again, this is a very basic question. It's what I asked the other companies last week. It just— this is a simple yes or no. Either the company has insurance liability in the event of a catastrophic risk around their AI or not.

I just don't want to misspeak. I do understand the question. I just— I don't have the specifics on our insurance policies. Okay. I'm going to pause for a minute and pass it to the chair for her questions and to other members at this point.

Thank you so much, Madam Speaker. Ms. Castillo Madrid, you're the senior manager of global government affairs. Are you an attorney? I am, yes. Great. And in your role, These questions that the speaker just asked in terms of, you know, the numbers, do you have access to that type of information? To be clear, although I am a barred attorney, I don't practice law at SpaceX, or I haven't practiced law in reality for almost a decade.

Okay. But do you have access to this type of information, like some of the numbers the speaker has asked for? I'm imagining as someone who's a senior manager of global affairs, You deal with regulators, you deal with elected officials on a worldwide basis. These are very simple questions that this council is asking. So my question to you is, do you have access to either yourself having direct data that you could give this council or the people who are in charge of having such information?

I would have to get back to the team and make sure that, that we have what you need or what you're seeking. Great. Hopefully Congress also calls you up to testify. And when they do, I hope you have those answers. You testified that risk assessments happen once a year and, and you said, and we report to the authorities when required.

What type of risk assessments happen and what have you reported to the authorities and when is it required? Like I said, we don't release any models that we don't believe are safe. So the risk assessment, that's like an ongoing risk assessment, but we do test our models well before release, and it's just a continuing process. Safety protocols have to scale with the capability of the technology, and the public can take great confidence that we're testing— that our testing occurs at every stage.

So from pre-training to deployment and beyond. We also keep watching systems after deployment and only release models we believe we can keep under meaningful human control. What have you— what have you reported to the authorities? What instances have been reported? I don't have the specifics on reports to authorities, but I will say that whenever it's required, we, we, we're ready to do so.

When is it required? By law. Have you reported any incidents to the authorities? I'm not aware of any incidents that we have had within the, the time frame that we're that we're discussing right now. But like I said, I can get back to the team and get you an answer. So what you testified is a protocol or a procedure that is in place in case it's necessary, but it hasn't been necessary to your knowledge?

To my knowledge, we haven't had any rogue agents or anything like that. Obviously, the speaker did mention something that happened earlier this year, and we made reports around it. And I even mentioned how we work with law enforcement, the National Center for Missing and Exploited Children. We have— we do have a great partnership with them that has been going on for a long time.

And we take very seriously our partnerships with law enforcement to make sure that any bad actors are caught and are kept off of the different platforms. And in your role, if something were to be reported to the authorities, would you be notified? I don't want to speak out of line. I think the legal team would be a better— So why is the legal team not here?

I'm here to answer your questions as best as I can, and I'm happy to report anything, anything that you need more of after the hearing. How many whistleblowers has your company fired? I'm not aware of any cases of whistleblowers. We have industry standard protections for the safety and no retaliation against whistleblowers. I just want to interject for a minute because Devin Kim, one of your former employees, was reportedly fired after he sounded the alarm about serious safety concerns.

Why was that? I'm not aware with that specific case. What I will say is safety is at the core of everything we do, and we take safety very seriously. And everybody has an ability to say something when they see something. We have even for before a launch, everybody receives an email from leadership saying, if you have any reason to believe there's a safety concern, please let us know, and we'll delay anything that's happening.

So like I said, we take safety very seriously, and we'll delay any release of a model. But if you take safety very seriously, why would Devin Kim, who sounded the alarm about major safety issues, which he brought to leadership, as soon as he did so, he was fired? I'm not aware of that specific case. I don't understand how you can say you're not.

I mean, you're under oath, so you, under oath, you're not aware of anything regarding Devin Kim? I'm not aware of that specific case, Madam Speaker. I mean, okay, it was widely— it's been in the news, it's been on social media. I just find that not credible. I'll pass it back over to the chair.

I mean, you just finished saying that the company sends company-wide emails before lunch, and something of this nature that has been reported, that this council already has the information, but yet you're an internal. agent of your company and you have no idea about this firing. It's hard to believe. And at the crux of this issue is trust and the public's trust.

And when you all come before us and do not answer our questions or purport to not know, it does— it erodes the public's trust. And that is part of the problem and how we got here. And it's also shameful that, you know, the speaker had to use the council subpoena power to even get you here. We could have done this last week for the answers that you're giving us today.

Does your company— well, first of all, let me acknowledge that we've been joined by Councilmember Narcisse. Thank you for being here. Does your company currently sell or deploy AI systems in New York City to employers for hiring, to landlords for screening prospective tenants, to lenders, to schools, to city agencies? No. Our models are available, are widely available to the public.

Do you sell them to the City of New York? I'm not aware of specific cases where we have sold or not sold. I'm not part of the team specifically, but I will say that all our models are widely available for the public. You're Global Government Affairs. The City of New York is a government entity. I'm quite confused about your role in the company.

Does your company sell or deploy AI systems to law enforcement agencies? We have great partnerships with governments across the US and internationally. Law enforcement agencies, the Department of Defense, any other agencies? That includes multiple agencies across US government and And do you know which ones? I don't have an exhaustive list of our partnerships with U.S. government, but we're proud to partner with, with them.

Is all of your cloud infrastructure located in the United States? If not, which other countries is your infrastructure located? I believe all of our infrastructure is located in the United States. Okay. Going back to the speaker's line of questioning around deepfakes and sexual images. And I'm sorry, Madam Speaker, that your image was used in that way.

You know, it's disgraceful. And we heard last week the very moving testimony of Jess Asado, who is a member of the Parliament in the UK, who is— my understanding is suing your company. Do you have any comments on, on Member Asado's case? Yes. To be clear, I'm a woman. No one should have sexualized images made of them without their consent, and our policies prohibit that.

Obviously, her case is pending, so I can't discuss the specifics there, but I do want to say we're all on the same side. Bad actors should not be allowed to do and go around the safeguards that we have, and that's why we continuously improve those safeguards and make sure that Grok is not used to exploit minors, commit crimes, or generate illegal content.

It was reported as of December that SpaceX has set aside $530 million for potential litigation loss, some of which could stem from ongoing complaints filed against its AI unit over sexualized imagery generated by Grok chatbox. There are several lawsuits filed against you. Do you anticipate more cases filed, although there are restrictions in place? I couldn't predict, like, the litigation.

What I will say is that we are strengthening the systems to ensure that nobody is going around our safeguards and using Grok for illegal activity. But you anticipated $530 million in losses. I'm not aware of the specifics on that. What I will say is I can't predict what will or not happen in that litigation. You know, the speaker was just sharing her own experience, and it is disgusting.

And, you know, you're a woman, and I appreciate you being here, but the fact that we could pull an image up like that this morning means that whatever safeguards in place and this line that you keep giving us about safety is a priority, that is not safety when a woman's body is being utilized in the way that these pictures are sexualizing the leaders of our city, that's not safety.

So in that case, how is this line about safety being implemented when just this morning we were able to pull up this, this picture? Yeah, and I just want to build on that. Thank you, Chair. Like, literally within 30 seconds, and that was not even on the spicy mode, this was on regular Grok. We literally asked regular Grok to pull up a sexualized image, and a sexualized image comes up, that is available for all to see, for my daughter to see, for anyone to see.

I just don't understand how you can say that there are these safety models when clearly it was less than 30 seconds to just pull up this image. All I can say is Grok has strict controls in place to prevent the sexualization of children and real people. I do apologize that you had to go through that. And like I said, I I don't think anybody should be put in that position, and our policies prohibit it for that reason.

To be clear, it's not that I had to go through it. I literally just asked Rock to pull up the image and it came up. I mean, this is now— it's not just about for elected officials, it's for anyone. So it's just unacceptable. And the other thing that I just want to say is that when you say— you keep giving us this canned answer about safety, safety, safety.

Getting back to Devin Kim, his title was president of the Center for AI Safety. He was one of the early employees at SpaceX AI. So it's just completely not believable that you are not aware of the circumstances around his firing. I understand your frustration, Madam Speaker. Unfortunately, like the chairwoman said, I There's a lot in that I cover, and I want to make sure that I am being as truthful as I can.

And I answer the question with, with the facts that I know. But if he's the president of the AI Safety, I don't understand how you could not be familiar with his termination. I'm not familiar with the details of, of, of why he's no longer in the company. Your company has challenged in court Minnesota's restrictions on AI tools that enable non-consensual Non-consensual sexual imagery.

Where do you believe legal responsibility should lie when a user employs your model to create such material? Is it with the user, the developer, or both? And what obligations should developers have to prevent the misuse? To be clear, we're not fighting a ban on non-consensual deepfakes. Our own, our own policies prohibit modifying real people's image without their consent.

Our concern with that specific law you cite is that the law extends to lawful content, and the 8th Circuit agrees, and they temporarily blocked enforcement because of that. So, in your opinion, it's not your job to restrict sexual imagery, just sexual imagery that's done without consent. But here we have, for example, the speaker of the city council who has not given her consent, for the use of her image, and yet we find it in 15 seconds.

So what responsibility lies with your company in your opinion? I mean, we support measures that protect the public from harmful synthetic media while allowing legitimate creative and scientific uses of our product. This is not a scientific use, FYI. I want to ask one more question before I turn it to some of our colleagues.

The White House Task Force on Superintelligence was established by presidential executive order and is intended to advance the application of existing liability laws to AI companies while directing the Department of Justice to prioritize enforcement of certain criminal statutes against those who use AI unlawfully. The task force held its first meeting yesterday. Do you support holding frontier AI developers legally liable when wrongful acts are carried out through their models?

If so, under what circumstances should liability rest with the developer rather than the user? As part of the White House Accords that you stated, we— one of the first ones is company responsibility, and companies acknowledge that they're responsible for developing technology safely. and for building public trust around it. Okay, I'll pass it on to Councilmembers Maloney, followed by Narcisse.

And we've been joined by Councilmember Gutierrez. Thank you, Chair, and thank you, Speaker Menin. We've spent some time talking about some of the potential misuses of the platform and that you have a safety team and that you're tracking these instances. But I'd like to dig in more specifically into cases where AI is being misused. What are some of the categories of misuse that you do track?

We've talked about some of them today, unfortunately, the sexualized deepfakes issue. There are instances of hate speech, including antisemitism. What are some of the other misuses that your team is tracking? So we have terms of service where we lay out all our policies and what is considered use or misuse of the platform. Our biggest concern here is making sure that we're catching bad actors that are trying to get around those policies and the safeguards that we have implemented.

So could you be more specific about what the largest categories of misuse are? We've talked about 2 so far. I'd love to hear from the experts, the companies, of what those misuses are that you're seeing. We engage in rigorous training throughout the model development, including pre-training, post-training, fine-tuning, and adversarial stress testing. That means we try to make sure that anything that we sort of input into the system, make sure that it's not violating any of our policies.

We have strict policies around a number of things, for example, for chemical and technological risks. We test with expert evaluations and red teaming before release, and we publish results in our model cards as well. If you have any specific questions on that, the model cards are very detailed on how and what risks were— or what our policies are focused on.

And do you track instances of misuse? We don't release models that we believe are not safe, and that includes misuse. But of the models that are We already talked about some cases of misuse. We talked about sexualized deepfakes. We talked about hate speech. Do you track the number of instances where those things are happening? Do you have data on that?

I don't personally have data, but we do have, for example, the ones that you mentioned. We do track them and make sure that when we catch any gaps in our safety, that we're filling those gaps with stronger protections. And we tighten those before, um, keeping the, the model in the public. And do you share that information publicly?

Could we get that information of the number of instances of harm? And there's 2 that we've mentioned, but I'm sure you're seeing other instances or cases of misuse. For example, privacy breaches is one I can come up with. Uh, and I'd love to get that level of detail, both the types of misuse that you're seeing and the volume.

Are these pervasive, or is it just a few instances? It sounds like it's quite pervasive, and not having that data is very difficult. If you can't track it, how are we to be able to ensure that these models are safe? Thank you, Chair. I'm at the end of my time. Was there a question at the end of that?

I'm sorry, I thought you were just making— I was out of my time, but my follow-up is just, to get information on the number of instances, the types of misuse, the things that you're seeing that bad actors are doing on the platform, and following up with the volume? The— are we seeing more cases of sexualized deepfakes, like having a sense of the scope of this problem?

And then the second question here, if the chair is giving me time here, is if risks are higher for open-source technology And how do you track those risks after a license has been granted to a third party? How are you able to see if they've stripped down the safeguards and built something harmful that violate your policies? How do you track that?

Understood. To address the questions, we cannot predict every way a bad actor will misuse a frontier model. What we can do is treat reporting and partnership as elements of safety and further foreclose the opportunity to misuse. And we believe better information to law enforcement lets investigators act. And our goal is simple. Like I said, we're all on the same side.

We want to make sure bad actors are not exploiting this technology. And we're here to make sure that our safety standards are kept at every step of the way. And do you think that those risks are higher for open-source technology? I don't have a particular opinion on that. But what I will say is that we make sure that our safety protocols are always in place when it comes to SpaceX.

So the cases of misuse are shared with law enforcement? When appropriate, I believe they do. All right. Thank you, Chair.

Thank you. Councilmember Narcisse. Thank you. X publicly encouraged people to upload their X-rays and MRIs to Grindr. for analysis. Where does that medical data go? And if it's used to train your models, and does any HIPAA-level protection apply? You said that you didn't release models you don't believe are safe, right? Correct. Was promoting for reading X-rays, MRIs, and I believe when NYU radiologists tested it on the breast imaging, it did not get any single diagnosis right.

Did that pass your safe, whatever the safety test we're talking about, thinking is it safe? Is it a standard we use for medicine? A drug does not go to our market, right? DEA will not approve it, right? So because of the company, if it's not safe for us, why should AI that gives medical advice be any different?

I don't understand it. To be clear, Grok or any of our models or SpaceX AI tools are that, are tools for the public. To be able to understand the world that they live in better. It's not meant to be a clinician, a doctor. It's never going to replace the important relationships you build in person with the people that are meant to take care of your health.

And in terms of privacy, users decide whether their conversations are used to train our models or not. So we leave that to, to make sure that they have a say into, into that. And they can turn it off at any times in their settings or use private chat, which is also available, and it's deleted from our systems within 30 days.

Okay. Is SpaceX AI selling or piloting any products in our hospital, in the city system, or insurance company in New York? Do you have any idea? If so, what is the, you know, the output before it reaches a patient? I don't have the specifics on any specific partnerships with New York City. What I will say is that SpaceX SI and SI in general has enormous capabilities, and we've seen that it's accelerating scientific discovery and expanding knowledge and creating new opportunities for human progress.

And that also applies to medical field will obviously abide by HIPAA laws or any laws, then, and that's meant to be corrected. But the average Groq user should not be using our tools to replace a doctor or any specific service provider that has information that they need to stay healthy and safe. You know, when you're encouraging people that's sick and telling them you have a new technology, and I'm saying the X figure telling people to do that, and you're not ready, and it's not safe to tell you because misdiagnosis is a big deal in a medical system.

And then you using it, telling people openly, on X to put their MRI x-rays and then thinking that this is a new technology, this is the world safer, you saving the world, but it's not safe. And you asking the— you telling our speaker, she's frustrating and, you know, asking the question, not getting answers. But right now people are utilizing the tools, especially the young generation, to expect a definite answer.

You know how depressing that can be for a person to think that your X-ray is wrong, your MRI is wrong, and you encourage people to do so? So that's the frustration. You're supposed to give us answers, but you're not. You're just being here. I will say some word, but I'm not going to say doing something to my brain.

Thank you. Thank you.

Councilmember Gutierrez. Thank you, Chair. Thank you, Speaker. Can you confirm if in the new set of bylaws shareholder class actions are barred? I cannot confirm that. Okay. I'm not aware of that. Not aware. Okay. Do you— are you aware of any other mechanisms or instances where limits to users' abilities to take action exist? Can you— I'm not entirely sure I understand the question.

So very plainly, are there instances where users are being challenged to hold your company accountable? I asked about the example of an inability to bring up class action suits. You said you cannot confirm. Hopefully you'll get back to us. But are you aware of any other instances where a user cannot hold your company accountable for whatever reason?

Are you aware of any of those challenges? I mean, we're responsible for following the law, and existing laws apply to SI, and using SI is not a legal defense. Are there instances where if a user feels that they have been targeted wrongfully, are there instances where they are— where they themselves are not allowed to bring the company to court, for example?

We're responsible for following the law and how it's written. So I, I don't, I don't necessarily understand where the distinction would be, um, in what you're asking. Well, I know you can't confirm, but my understanding was in the new bylaws shareholders cannot bring class action suits. So that's why— that's what I was asking. But hopefully you can confirm and share that with us, uh, very soon.

Um, I have a couple more questions just related to personal data. One is specifically, can your company know, or do they have reason to know, that any user is intending to perpetuate violence in any way? My understanding is that your company can profile users based on different prompts, searches. Can you share if there is any way for your company, using those same mechanisms, can you determine when a user wants to perpetuate violence in any way?

Yeah, like I mentioned before, we want to catch the bad actors and we have a combination of AI and human monitoring to make sure that we're getting those bad actors off our platform and hopefully into the authorities when necessary. And can you share what, what is the process for communicating to local authorities? We have a law enforcement, the way that people can sort of report.

I can give you more specifics after, but anybody can report and we all take very seriously all the reports. Is the reporting just calling 911? I meant on the platforms, not— obviously, anybody is free to call the authorities in that regard, but For example, with CSAM, we work with the National Center for Missing and Exploited Children, and we make reports to them, and then they have authorities from different law enforcement agencies across the country, federal and local, and they are in charge of following through and making arrests when necessary.

Okay, Chair, I have one more question. Sorry. Thank you. My next question is just related to a user's request to delete personal information for whatever reason. Can you share the process for a user who is asking for their information to be deleted? And can you guarantee that there is a confirmation that, yes, all of their personal information has in fact been removed?

Yes, we take our users' privacy very seriously, and we give them control over whether their Grok conversations are used to train our models or not. And they can personally turn it off at any time in settings or use private chat. When they use private chat, obviously it's not used for training, and it's deleted from our systems within 30 days.

It's removed within 30 days? When using private chat, yes. Okay, thank you.

Thank you. Okay, last night Anthropic issued a statement that they are committed to having evaluators assess their safety practices and giving employee-like access to a team from Model Evaluation and Threat Research, not-for-profit evaluation group. Elon Musk then responded, replied that SpaceX would, quote, probably do the same, end quote. Is that the position of the company? As part of the White House Accord, ongoing collaboration was part of it, and regular meetings among companies to establish standards and best practices are very important.

This is an evolving technology, and safety is at the forefront of everything we do. So we want to make sure that we're doing everything we can to keep our systems So is that a yes or a no that you would have agreed to this kind of third-party validation? Independent external evaluation was also part of the agreements and just getting partnerships with independent external auditors and evaluators to assess model capabilities, controls, and safeguards.

So do you agree that for SpaceX AI, or really for any AI company, when they commit to third-party validators, they must be independent, and that the company needs to give those validators high-level employee-level access, along with all the appropriate and necessary information so they can truly do the work to validate? Yeah, partnership with independent external auditors and evaluators to assess safety is part of what we already agreed on.

One legal expert that we had at our hearing last week testified that Whoever keeps a tiger is liable for injuries or deaths when it escapes, however strong the cage. His point obviously was that governments often impose strict liability. I know you indicated you are a lawyer. Governments often impose strict liability on abnormally dangerous activities because even reasonable precautions cannot eliminate the risk of serious harm.

This expert testified that frontier AI development belongs in that category since leaders of the major AI developers have acknowledged that they don't yet know how to align the most capable systems so that reasonable care, that standard of reasonable care cannot truly eliminate the risk. Do you agree with that premise? Yeah, we're responsible for following the law and existing laws apply to superintelligence and using superintelligence is not a legal defense.

So you believe the existing laws are sufficient and that new laws are not necessary to regulate AI? We're open to working with policymakers like yourselves and others around the country and internationally to make sure that we're putting in place common sense, technically feasible regulations. So if a SpaceX AI agent harms a New Yorker without any individual SpaceX employee acting negligently, who— how should victims get compensated?

I wouldn't get into the legal analysis right now. I don't believe I'm equipped to do that. But what I will say is that we're responsible for following the law and all existing existing laws applied to SI. But so, for example, do you— would you think it's acceptable that perhaps a small business that is devastated by a rogue agent or agents that wipes its customer database, that they have no recourse other than— you keep talking about the existing laws, but that would be a complicated, expensive negligence lawsuit.

I understand. I wouldn't want to get into hypotheticals. What I will say is We're proud to follow existing laws and working with policymakers to make sure that we have technically feasible and useful regulations. Okay. I'm going to pass it back over to the chair.

Thank you. Would you agree that, as Mr. Kim warned, developing and maintaining safe AI involves identifying potential risks such as bioterrorism, AI-enabled cyber warfare, and rogue power-seeking systems, and implementing processes to avoid and mitigating these risks? Yes, we train our models to ensure they don't assist with dangerous activity. We evaluate them for misuse and attempts to get around safeguards and loss of control, and we monitor them after deployment.

I also mentioned specifically for chemical and biological risks, we test with expert evaluations and red teaming. before release, and we publish all these results in our model cards. What are some of the procedures in place if there is that type of risk? To that, SpaceX AI engages in regular trainings throughout model development, and that includes pre-training, post-training, fine-tuning, and adversarial stress testing.

And that We do that, and it's not something that stops once the model is released. We understand that it's important to make sure our safety protocols are scaling with our capabilities, and the public can take great confidence in that we're testing— the testing that occurs before models are released and the safeguard stacks that surround the model deployments are ongoingly, or we ongoingly look for for any issues and make sure that we're closing any gaps that need stronger protections.

Your counterparts at the other AI companies, when we questioned them about the Hugging Face incident, for example, you know, what we learned at that hearing was that, you know, the agents kind of escaped the sandbox and there was incidents where they were kind of acting on their own. Have— has SpaceX I'm not aware of any cases where any of our agents have gone rogue.

What I will say is that we believe in rapid, accurate notification of significant safety incidents. We understand that reporting is important, and we investigate issues and share information with appropriate parties when it does not create additional risk.

Do you believe that since you all changed your name to SpaceX now, how close are we to superintelligence and when, in your opinion, can we realistically expect it? I wouldn't weigh in on that. I'm unfortunately not an expert on that. I wish I could speak more intelligently on what we can predict in the future. What I will say is that We want to make sure that our safety is at the forefront of everything we do and that with everything that can be accomplished with superintelligence, that we are staying on top of it, making sure that everything from pre-training to release and beyond, we can stay in control of our systems at every stage.

Given that your company is so big on, you know, safety, Although we're not considering any bills at this hearing, your company has challenged several state laws, including California AB 2013, Generative AI Training Data Transparency, Colorado SB 24205, Colorado Artificial Intelligence Act, Minnesota's law, which we talked about. If the council passes a law requiring AI systems sold for use in high-stakes decisions to undergo independent third-party validation, before deployment.

Would your company comply with that requirement, challenge us in court, or stop offering those systems in New York City? I'm not going to predict the future, but what I will say is that we comply with local, state, and federal laws and advocate for policies that are technically feasible and provide meaningful safeguards for users. Yeah, but you've also challenged these laws in other states.

And so if your company is so big on safety, this council, under the leadership of the speaker, has identified these third-party validators critical component of that? Yeah, unfortunately, when laws are written, they have different components to it. So I can't speak to like everything that's challenged and the reasons why they're challenged. But what I will say is that our commitment to safety is at the forefront of everything we do.

I just want to challenge that because if your commitment is to safety, why did we have to subpoena the company to compel you to be here? I'm here today to answer your questions as best as I can. I wasn't part of any conversations before that, and I'm happy to be here and help with anything I can in the— It just doesn't instill a lot of trust and confidence when we literally had to— not only did SpaceX AI fail to come to our hearing, we had to issue a subpoena and we're ready to go to court to enforce that subpoena before we were able to secure the company's commitment today to testify?

We submitted a written response and have been in contact with counsel and your counsel's counsel since last week, and we agreed to appear here today. And I'm happy to answer— continue answering your questions as directly as I can.

So we have numerous bills that we heard last week at the hearing that the company did not attend. Is— will SpaceX AI commit to support the council's legislation? I believe the 10 AI bills that New York City Council is proposing are under review by the team, and we're happy to work with you to make sure that we are advancing technically feasible and any— provide meaningful feedback on what safeguards need to be Do you have more questions?

You're good? Okay. Does any other councilmember have more questions? Okay.

We thank you for your testimony today. We obviously have a tremendous amount of remaining questions. I think a lot of the questions you did not answer, and so we are expecting that you will provide that information that you did not provide. Again, it is Our goal, again, is not to stifle innovation, not to support the incredible medical and scientific breakthroughs of AI, but to ensure the public that these serious safety concerns that have repeatedly been raised in recent months are being addressed and are being addressed in a thoughtful way, which we strongly believe our legislation, particularly the third-party validation free of any conflict of interest would help to assure the public.

In addition, I want to say I'm deeply disappointed by the comments about Devin Kim, because it is very clear there are numerous articles out there that— about his firing, and the lack of answers and transparency about that are incredibly concerning. Thank you, Madam Speaker. Okay, thank you. Thank you so much.

Um, the next panel is Devin Kim on Zoom, and I apologize in advance, but the next person is also Keojing Ella Zheng on Zoom. And please correct your name for the record. And let me just say one thing. We are very pleased that we have Devin Kim here today. This This was something that was secured this morning.

We really appreciate Mr. Kim's willingness to testify. This is something very, very important. So, Mr. Kim, we are really pleased that you were able to join us today. Thank you. We'll start with you, Mr. Kim. Thank you.

Good morning, Chair De La Rosa, Speaker Menin, and members of the committee. My name is Devon Kim. I'm the president of the Center for AI Safety. And before that, I was one of the first employees at xAI, the company that we're hearing from today. And I built the systems that researchers use to train and evaluate Grok. I'm suing xAI for retaliation, so I'll stick to my public complaint.

Starting in January 2025, I raised concerns that Grok could assist people with the creation of bombs and bioweapons, and I asked for proper biorisk safeguards. My supervisor responded that AI safety was useless and that AI would kill us all anyways. We launched models without those safeguards despite warnings from me and many other trusted employees. In August 2025, my supervisor pressured employees to circumvent European safety regulations.

In September 2025, he fired me the same week I was scheduled to present safety— my safety findings to leadership, and most of the safety team resigned after that. When a model is unsafe, the first people to know are the employees inside the company. We see it in the evaluations and in leadership decisions months before anyone else finds out.

And right now, those employees have very little reason to speak up and lots of reasons not to. There are 2 things that I'd ask of the council. Please protect people who report dangerous conduct even when it isn't illegal yet. Almost everything I raised was dangerous before any law covered it. And count a loss of control or an unauthorized access as a reportable incident before anyone gets hurt.

Under the RAISE Act, an incident only counts once someone is killed or injured. So neither Hugging Face nor the Australian Medicare breach would have qualified. I think that's a reasonable place for the city to start. Thank you, and I'm happy to answer any questions today.

Thank you so much, Mr. Kim. I don't know if your attorney had any statement that they wanted to make, or can we begin the questioning?

Yes, Madam Speaker. Good morning. And good morning to the members of the City Council. My name is Ella Jung. I'm a partner at Stanford Heisler Sharp& McKnight, a national civil rights and public interest litigation firm. We represent employees who have the courage to speak up against powerful corporations, including whistleblowers who expose serious threats to public safety. Thank you for the opportunity to testify along with my client, Mr.

Devin Kim, about one of the defining challenges of our time. Ensuring that the race to develop artificial intelligence does not come at the expense of human safety. Local, state, and federal governmental bodies have abdicated their responsibility to protect society. Whistleblowers who raise existential threats to humanity often pay the ultimate price for doing so. Those whistleblowers need protection, and you are in the body who can provide that protection.

This body, the New York City Council, has an urgent obligation to protect the employees of New York City who raise issues designed to protect the public and who act in the public good. There's no more urgent matter facing the City Council, and indeed every city council state government, and federal bodies in the United States. We are already at a point of extreme vulnerability.

Mass destruction, societal chaos, and extreme danger are all potentially minutes, days, or weeks away, as you have heard during the October 5th hearing. AI has extraordinary potential to benefit humanity, but its rapid development also presents extraordinary risks. And the people who often know the most about those risks are the engineers, researchers, and safety professionals working inside AI companies.

Unfortunately, like my client, speaking up can come at an enormous personal cost. Our firm currently represents Mr. Kim in his whistleblower retaliation lawsuit against xAI and SpaceX. As alleged in our complaint, Mr. Kim raised serious concerns about AI safety and compliance related to Grok. Rather than having those concerns properly addressed, Mr. Kim suffered retaliation and ultimately lost his job.

Mr. Kim's case illustrates that we cannot rely on AI companies to self-regulate, and we should not allow them to harm individual employees who are willing to sacrifice their careers and livelihoods to protect the public. In my law firm, we routinely represent whistleblowers like Mr. Kim who stand opposed to illegal and/or unethical behavior and individuals harmed by AI, including executives and employees at companies who are retaliated against for speaking out about corporate wrongdoing.

That corporate wrongdoing often involves AI fraud, digital ad fraud, and misrepresenting cybersecurity risks and control. Engineers, researchers, data scientists, and trust and safety staff at frontier AI labs who raise concerns about model safety, testing, deployment, and data practices, and who are often the first to be punished for it. They are often fired, demoted, sidelined, or pressured out after raising concerns with their employers.

Victims of cybersexual abuse. These victims have their intimate images posted on social media and manipulated by chatbots, or are stalked, surveilled, spoofed, or deepfaked. Employees, customers, and users of technology who are victims of privacy and data breaches, which include unauthorized collection of biometric, and employee data and unconsented use of personal information. In addition to those areas, whistleblower protections should also extend to holding tech companies accountable for the harm created by their products, including AI tools— how it was designed, what safeguards were built into it, and whether those safeguards were implemented and activated.

Whistleblower protections should also extend to incidences of algorithmic biases. Employers use software to screen resumes, rank candidates, score video interviews, set pay, assign shifts, and decide who is promoted or let go. When those tools produce outcomes that disadvantage people because of their age, race, sex, sex, disability, or another protected characteristic, the employer should be held liable. Meaningful whistleblower protections are essential.

New York City has an opportunity to help establish a model for responsible AI governance. We must ensure that the people who come forward are protected from retaliation, that their identities remain confidential, and that the law reaches serious AI safety risks that may not fit neatly within existing regulatory violations. We can learn from successful whistleblower programs at the federal level, including the SEC's, which demonstrate how meaningful financial incentives can encourage individuals to expose wrongdoing that might otherwise remain hidden.

We can also learn from California, which has taken an important step through SB 53 Signed into law last year. That bill established transparency requirements for frontier AI developers and protections for employees who report serious AI safety risks. Can you conclude, please? Yes, we have some questions. Thank you. Yes, of course. Therefore, I encourage the council to establish 3 essential safeguards for AI whistleblowers.

One, strong anti-retaliation protections with meaningful remedies. 2, secure and effective reporting mechanism. And 3, financial rewards that recognize the extraordinary risks that whistleblowers undertake. No whistleblower should ever have to sacrifice their future to protect ours. Thank you for hearing me out, and I'm happy to answer any questions. Thank you.

Thank you both for your testimony today. I do have a number of questions. So, Mr. Kim, you You mentioned that one of the things that you really want to see is strong whistleblower protection. So our bill, for example, has a financial reward for whistleblowers and gives the whistleblower a significant portion of the financial penalties, which can result in the multimillions of dollars, since it is a per-agent, per-instance penalty.

So if there are thousands of rogue agents or the product is being sold and marketed in New York to tens of thousands of people, these— obviously these penalties will add up. So I appreciate your comments on that. Also, the idea of the protection for the whistleblowers This is exactly why, at our hearing last week, when we had Alex Turner here, we issued a friendly subpoena so that he could freely testify.

So to that end, when you were employed by SpaceX AI, did— was there any nondisclosure or confidentiality policies that could be interpreted as restricting employees from reporting safety concerns? I'm not a lawyer, but yes, all employees were required to sign an NDA on joining the company. And were you ever instructed that if you were to speak out or raise certain safety concerns, that would be something that would violate the NDA?

Not explicitly, but I believe that was strongly implied. and the way that the company was run. And you allege that you repeatedly warned SpaceX AI leadership that Grok could facilitate the creation of bioweapons and other dangerous activities. Can you describe the specific safety gaps that concerned you the most and what safeguards you might have recommended before deployment?

Yes. Typically, before companies release models, they undergo safety training. And they also undergo a specific type of safety training called refusal training. Models are trained or should be trained to refuse dangerous queries, such as assisting in the development of bioweapons or other weapons that would be very harmful. And this is also notably the kind of information that is very difficult to obtain today and in other means.

And at the time when xAI was releasing, um, One of their earlier, or I should say several models, we did not have those safeguards in place. So you mentioned that you had discussed these safety concerns with your supervisor. Were you ever able to issue these safety warnings higher up to more senior leadership before you were terminated? I was, and this was also a very controversial issue within xAI that many employees and researchers were very unhappy with.

And based on your experience inside the company, what minimum testing, evaluation, and red team procedures should frontier AI developers be required to complete before releasing any model to the public? There are a number of very critical safety gaps that are present in xAI's model releases that are not present in other companies. One gap is critical thresholds. Companies typically commit to not releasing models above what they consider to be critical thresholds Whether that is in bioweapon capabilities, cyber capabilities, and other dangerous capabilities that the model will have.

A second part of these releases that is very critical is a documentation justification for deployment. Typically what that looks like is explaining what sort of critical thresholds were reached and what work was done to mitigate those thresholds. And their justification for why they believe this model is safe before releasing it. And then especially— and then a 3rd critical piece that is becoming increasingly relevant is companies should be testing for loss of control and manipulation capabilities of these models before releasing.

And specifically, these are standards that many companies have today but are notably absent and the XAI safety releases.

Gabriela Castillo Madrid, the senior manager of global government affairs at SpaceX AI, who testified before you did, I don't know if you had the opportunity to hear all of her testimony. But when I asked her about the circumstances around your termination, she said she was not aware of any of the circumstances. Is that a— and she was under oath.

Is that a factually accurate statement that she said, if you, if you know that is. That would be difficult for, for me to know, but that is a surprising statement, to say the least. And I don't know if you had the opportunity to hear the rest of her testimony. She kept saying that the company is dedicated to safety.

She kept saying that that was one of their top priorities. What did you think overall of her testimony? Well, as an early employee at xAI, I worked on the systems that researchers used to train and evaluate Grok. And because of that, I had a very critical part in developing Grok and had held close relationships with many of the researchers.

And I was trusted with many important responsibilities within the company. And I use that standing to advocate for safety and other important things that I believe that the company should be doing. And so I will note that if it was difficult for me to speak up and make changes that I believe were necessary within the company from my position, then I can only imagine for other people what that might be like for them.

I will also note that during my time at the company, I believe that this is an important problem to be working on and therefore I spent significant time towards this effort in a personal capacity. However, during my time there, there were only 3 employees working on AI safety, many of whom left after I resigned. And I think in large part also due to the frustration with how safety was being handled.

I can't speak to how AI safety is handled today. It's been a year since I was an employee at the company, but if it's anything resembling what I personally experienced, then I would say that's a very troublesome state of affairs for them to be in.

So you're saying that a year ago, there were 3 safety employees in the company working on safety around AI? Yes. And in comparison to at that time a year ago, how many employees could— can you answer where they're at the company overall? Well, it really depends how you're, how you're counting. We had a large number of AI tutors who were responsible for training the model, and they were contractors.

We had a large number of data center employees. I believe that in terms of the technical staff that were involved in the direct training and development of the model, the number that I recall is in the 200s. So the idea that there were only 3 people dedicated to safety, and when I asked Ms. Castillo-Madrid how many employees were dedicated to safety, she told me she could not answer that question.

Is that credible, that the senior manager of global government affairs at SpaceX AI would not actually know how many safety employees would exist at the company? It would be difficult for me to speak to that. I will say it— what is credible is if she is not deeply involved in understanding the day-to-day of the safety development of the company.

Whoa. I say this because I was— I had strong relationships with the 2 formal legal counsels during my tenure at xAI. The reason we had a strong relationship is because I would point out issues going on within the company or with the Grok model, and I would also propose and implement solutions to be solving those problems. And we were able to develop a strong relationship that way and making their jobs you know, jobs easier and helping them, helping the company do the right thing.

I think given the level of detail that I heard today, I'm not— I wouldn't feel comfortable that she would have the same level of detail and understanding. 2 more questions.

Is the company currently, to your knowledge, validating the models with outside third-party validators who are free from conflict of interest? Not to my knowledge. And lastly, I don't know if you heard the earlier part of the questioning, but I pointed out that within 30 seconds this morning, I was able to ask Rock to do a sexualized image of myself, and there was an uber-sexualized image of myself under my official portrait.

And I question Ms. Castillo-Madrid about why is this continued to be allowed by Grok, this idea that within seconds these images— and this was, this was in just the regular Grok free app, not even the spicy mode, this was just regular Grok— within less than 30 seconds, an image that if my daughter saw would be horrified. Thank you.

If other— and this is just something that's readily available. Why do you think that SpaceX AI is continuing to allow this to happen? Well, I wouldn't be able to speak to the standards today. However, the standards and directive that were shared with us when I was an employee was that tasteful nudity is acceptable and is the goal.

I now will note that, you know, the tasteful is subjective, and there was no clear person designated as to what the line should be there. The issue that I have with that is there was nothing tasteful about the image that we found. And in fact, you could easily ask Grok to make it more sexualized. And so that was a prompt that Grok accepted.

And we had successive images, each of which would become more sexualized, more offensive. So this idea that it was, like, tasteful nudity or tasteful is just, you know, preposterous. But Mr. Kim, thank you. I really want to thank you for being here today, and I'm going to turn it over to my colleagues who I know have questions, but I just want to thank you for your courage and bravery in coming forward and in speaking with us today.

And now I'm going to pass it over to Chair DeLaRosa. Thank you, Speaker. And I want to agree with her. It takes a lot of courage and bravery for you and, and your attorney to be here as well and really give us some insight into your experience, which is horrific. I want to ask you, in 2025, one of AXI's co-founders indicated that he would rather, quote, release unsafe— an unsafe model than a poor-performing one.

Do you believe that that is still the priority for xAI in the development of their models? I can't speak to the priority, but what I'm able to share is that especially in the early days of development, I was told that because the company was in a position where we were losing or had worse models than a lot of our competitors, it would be difficult for us to focus on safety.

And that once we were able to have a more competitive stance, then we would be able to focus more on safety. And so at the time, that is a position that made sense to me and something that, you know, I really believed in. And so myself and many other employees, you know, I think a lot of people want to see the work have positive impact.

I think many people want to believe that they are the good guys. And if they're the ones developing powerful AI, that they would be doing a better job or a more moral job than the others. And so I think many people marched on with that belief. However, in practice, I think what we saw, you know, after many, many months of sacrifice, I mean, personally for me, I lived at the office for 4 months.

Because I was, and, you know, working at all hours of the day, including weekends, because I was, I was so dedicated, uh, to the mission. Um, once we launched Grok-3, which at the time was, was actually the most intelligent, uh, model in the world, um, and so powerful and so surprising to many of our competitors that, um, OpenAI reportedly declared a code red and sort of trying to understand how Exa had caught up so quickly and trying to do something about the situation.

Even at that point, it was very difficult to advocate for safety. Hiring for safety headcount was being blocked. And so I think that is the point at which many people started having the realization, oh, maybe what was told to us before isn't actually the case. Do you believe that other employees, to your knowledge, who were maybe afraid to speak up or were challenged in the same way that you were challenged when they did speak up ended up being terminated as well?

I wouldn't be able to speak to that. And in terms of what you've witnessed now that you're on the other side of this trying to create safety around AI, do you believe that anything has changed at Grok I wouldn't be able to speak to that. Okay. One of the things that was, you know, very alarming from your testimony is that you said that when you raised this up to management, they said safeguards are useless and AI will kill us all anyway.

You know, that is a very real statement to be made from a company who just testified under oath to this council that safety— I mean, I think that if, if we had a ringer every time she said safety, the bells would still be ringing here in the council. Safety is the priority. We do not deploy models without safety.

We ensure that there is safety. There is always a human being who's in charge of safety. Thank you. They said it in so many different ways. But yet you're testifying today that your direct supervisors who understood the work you were doing said AI would kill us anyway. Do you believe that safety is really at the heart of what this company is projecting?

Well, I will again note that there is a significant gap in the model card or sort of safety documentation that in the releases that xAI puts out. And I think the sort of explanation that, well, we started later and we haven't had as much time doesn't really work anymore, given that Meta, who, you know, started later in the AI race and was able to build up a team fairly quickly, do have the higher safety standards that I mentioned earlier in my testimony.

Yeah, understanding legal constraints, and maybe you weren't working on this, but another part of her testimony this morning was about how, of course we're safe, we're launching rockets. Do you believe that their safety protocols are in line with regulations for the type of space exploration and other things that the company's exploring at this time? I believe that the current set of laws and regulations around AI development are deeply inadequate.

Great.

Thank you for your questions. Council members, we're good. All right. Well, thank you so much for your questions and again for your bravery today and for being here. We really appreciate you giving us insight and understand that there is trauma associated with your departure from the company and the way you were treated. And so we really appreciate you giving us some insights and helping us to work around our legislative process to make sure that other workers are protected in this space.

Thank you for being here. Thank you. Thank you. Thank you both. I now open the hearing for public testimony, and I remind members of the public that this is a formal government proceeding and that decorum should be observed at all times. As such, members of the public shall remain silent at all times. The witness table is reserved to people— for people who wish to testify.

No video recording or photography is allowed from the witness table. And furthermore, members of the public may not present audio or video recordings as testimony, but may submit transcripts of such recordings to the sergeant-at-arms for inclusion in the hearing record. If you wish to speak at today's hearing, please fill out an appearance card with the sergeant-at-arms and wait to be recognized.

When recognized, you will have 2 minutes to speak on today's topic, which is the continuing discussion about the risks posed by artificial intelligence. If you have written statements or additional written testimony you wish to submit for the record, please provide a copy of that testimony to the sergeant-at-arms. You may also email written testimony to testimony@council.nyc.gov within 72 hours of this hearing.

Audio and video recordings are not accepted. Thank you. I now call up the next panel. Megan Lagerquist, Bryce Rogers, Gary Oysteven, Kalima Amdakar— and I apologize, please correct your names for the record— and Stephen Kramer. Good afternoon. Yes, you can begin.

My name is Bryce Rogers. I'm co-lead of the Resisting Automated Disempowerment Collective, or RAD Collective, a grassroots New York City community advocating for city and state policy that protects human life, joy, and democracy as AI and big tech grow more powerful. We thank the council for reconvening today to finish the job that started last Monday. RAD wrote a letter to the council with feedback on all 10 introductions, but today my colleague Megan and I want to share more granular recommendations on 2 introductions in particular: Councilmember Murano's chatbot privacy bill and Speaker Menin's third-party validation bill.

But first, I'll straightforwardly share 2 strategic postures RAD assumes in sharing these recommendations. One, our goal is for AI tools served to New Yorkers to be safe and also to slow down the rate of advancement of frontier AI given the recklessness and rapid power concentration we're seeing. 2, our angle with our recommendations today is for this legislation to pass and for companies to comply with it, as opposed to something like signaling or intentionally extreme compliance burdens to trigger some sort of, like, legal battle or something like that.

I'll speak on affirmative consent in Marano's chatbot privacy introduction. Many users share deeply personal information with rich context when using AI products. Anthropic, OpenAI, Google, Meta, Amazon, Microsoft, and SpaceX AI by default collect user chat data and train their models on it. Some of these companies, like Meta and OpenAI, use personal data for targeted advertising. And Google, Meta, Amazon, and Microsoft combine this data with other information they've collected across their platforms to build a more complete profile of the user.

This is invasive. Others, like Anthropic, insist they won't sell user data. But as long as this is voluntary policy, they can walk it back anytime they need the revenue boost. When Anthropic initially launched Claude, they did not train models on user data and automatically deleted it after 30 days. In September 2025, they rolled this back to an opt-out regime.

We recommend paring down Murano's bill to focus on the core requirements, of affirmative consent for data collection, storage, use, and sale. It should also be expanded to cover data collected by agentic AI, not just chatbots. This ensures the strongest version of the most impactful consumer protections can move forward into law as soon as possible. Thank you, and I look forward to you hearing from my colleague Megan.

Thank you. And please, um, submit the testimony as well so we can review. Sure thing.

Hello, my name is Megan Lagerquist. I am also a member of the RAD Collective, and I thank the council for hearing my testimony today. The second policy we want to address regards third-party audits, which Speaker Menin has proposed through Introduction 1100. RAD supports independent evaluations of high-risk frontier models defined by the RAISE Act. Currently, many AI companies voluntarily work with AI third-party evaluators to run safety evaluations.

However, the companies very much control access to what information they get and can decide to move forward with development or release regardless of the findings. Meeter employees who investigated the OpenAI hack of Hugging Face reported that they were not provided with sufficient time to fully evaluate the incident, and they only had logs from a constrained time window.

To ensure that these products are safe for consumers, we must require independent verification that these models are sufficiently aligned with human instructions, do not engage in social engineering or manipulation, and that the company's safeguards and monitoring and containment systems are equipped to control their own products. We cannot rely on self-published model cards and even whistleblowers for full transparency.

Evaluators need to be guaranteed access to all documents and models they deem necessary. necessary. They need to be able to run their own experiments, and that means more than just accessing chain-of-thought logs, although human-readable chain-of-thought is important to preserve. They have to be able to see inside the model architecture to truly be effective at alignment auditing. And if they say that the models do not meet New York's legal thresholds for safety, these models should not be allowed to be sold to New Yorkers.

In order to effectively facilitate and enforce this regulation, the city needs to invest in expanding the capacity safety of the Office of Technology and Innovation because, as we heard from Cyber Command's testimony on October 5th, they do not have the expertise to regulate industry as it stands. There are independent AI safety research organizations that could fill the need for evaluators, but the council needs to build the legal support system to certify and empower them.

I will say the federal government is content with voluntary commitments. and no verification. But I believe that this council is not content, and I urge you to move forward with strong public oversight of the AI industry. Thank you. Thank you so much.

Thank you to the City Council for this opportunity to speak. My name is Kalima, and I'm here with Demilitarize Brooklyn Navy Yard, which is a grassroots campaign to get weapons manufacturers off of public land. The DBNY campaign is here to hold the Brooklyn Navy Yard accountable for its role in using AI to further the U.S. military-industrial complex.

Companies like Amazon, Google, and Apple make millions of dollars through contracts with the Department of War that are used to spy on communities abroad and at home. AI is not the future. It is another tool of the rich to exploit and surveil everyday people. The Brooklyn Navy Yard uses public resources to lease to tech companies that use AI to develop weapons.

One of the most recent tenants is Radical AI. This company uses artificial intelligence to find high-entropy alloys that can make metals for hypersonic missiles. These missiles will further exacerbate the climate crisis, not to mention the human cost. We have watched schools and homes be bombed by America— be destroyed by American bombs. We cannot turn away as our tax dollars and public land are used to destroy communities across the globe.

The tech industry is incapable of regulating itself. We see the cozy relationship between the tech industry and the U.S. military, as well as the increased use of AI in surveillance of racialized communities by entities like the NYPD and ICE. DBNY wants these fascist tech companies off of our public land. This is not what New Yorkers want our tax dollars to be paying for.

The Brooklyn Navy Yard's role in housing tech companies that work with the U.S. military makes them complicit in the war crimes that the U.S. is committing. DBNY is asking this committee to do its job and to examine the tech companies leasing public land at the Brooklyn Navy Yard to ensure that they are actually in line with the Navy Yard's mission to support the greater good instead of contributing to ecocide, genocide, and deportations.

Our public land should be used to provide community resources services, and education, not to create new technologies that will infringe on our human rights. Thank you.

Thank you so much. Good afternoon, or morning still. I'm Steve Kramer. I own the firm Get Out the Vote. I handle 59 members of Congress. I handle 8 governors, including our current governor. 3 years ago, I sent out an automated call that mimicked the voice of the president. It was a warning shot to the entire industry, as well as to America, about what's coming, about what was in the future at the time.

Little did I know that it wasn't the future I was predicting, it was the present at the time of things that were already going on in the mainstream all over the place, whether it be against individuals and sexualizing them, whether it be against politicians, including our mayor and other people here in Manhattan that were affected. But I am going to ask you all to think to the future now, because what we're seeing right now happened 3 years ago.

What we're seeing right now was preventable, is preventable. But what we're seeing right now isn't AI. AI is many years ahead of us right now. And the fact that it's being governed by companies that aren't really governing it and by itself, meaning the artificial intelligence, means that the bad actors that I was accused of being Aren't even human, and the safeguards that you put in here—not just today, but in the present—have to predict the future, and that's a very difficult job for anybody.

I sympathize with both what's going on in the hearing today as well as the former hearing. I hope that you think ahead when making your next steps. Thank you. Thank you so much. The panel is excused.

Oh, yeah. Okay, we're moving on to the Zoom panel. Um, the first person is Sophie Cope, followed by Ajit Zeng, followed by Chaudhry Hazan Mahmood. Sophie, you may begin. Hi. I spoke last week at the previous hearing, although, Julie, I think you walked out right as I started speaking. So I'd appreciate if you watched my message about the harms of AI to women.

In the context— I talked in the context of OpenAI and Anthropic. Evidently, based on what we've heard today, SpaceX is the poster child of such harms. I think the council and I are in full agreement that the answers provided during today's hearing were insufficient, given that other AI companies make these harms much more difficult to enact. But I don't want SpaceX to be a distraction to the wider issue.

SpaceX is wonderful for the larger AI industry because it provides a scapegoat for labs like Anthropic and OpenAI, which can use the comparison to SpaceX to appear more reasonable. Grok, ChatGPT, and Claude. all reflect the men whose companies built them. A pervert, a liar, and a crybaby. The differences between them show that the AI industry is already self-regulating.

I ask again to the City Council, please take this opportunity to let New York City be a beacon to the world in wresting power from these men. Thank you for your time.

Thank you. Ajit Singh, you may begin.

Thank you. I was just waiting to unmute. Thank you, Speaker. My name is Ajit Singh. I am a physician, informaticist, and public health practitioner in the H&H Safety Net system. I'm profoundly concerned with the expansion of artificial intelligence tools which enable corporations and government agencies to collect, analyze, and exploit information about ordinary people on an unprecedented scale, often facilitate discriminatory pricing, enable intrusive workplace monitoring, monitoring and expose vulnerable communities to predatory targeting and violence.

I'm urgently furious with how generative AI systems, including Grok, have enabled the proliferation of non-consensual deepfake imagery, child sexual abuse material, sycophantic manipulation of users, and mis- and disinformation undermining, undermining New Yorkers' ability to distinguish truth from false. New Yorkers are resilient, but New Yorkers should not have to endure what our elected leaders have the power to prevent.

We must identify who bears responsibility, establish enforceable safeguards, and hold accountable those who violate them. Companies such as SpaceX AI, whose empty testimony was presented before the council today, must be compelled to transparency and accountability, disclosure, and accept responsibility when their technologies cause harm. They're not trying to catch the bad actors. SpaceX AI is a bad actor.

I welcome a few of the council's recent legislative proposals. Introduction 1116 would establish a private right of action against AI developers However, the effectiveness of such measures depends on how the law defines these safeguards, who evaluates those safeguards, and whether New Yorkers can realistically access the courts. Although this can provide a formal mechanism for redress, the fact that individuals would bear the financial and evidentiary burdens means that in practice it'd be out of reach for most— for the most vulnerable to technological exploitation.

Thank you. Per Intro 1101, it would enable New Yorkers to report when individuals and groups violate AI-related laws directly to the Department of Consumer and Worker Protections, driving investigations. Your time has expired. If you could please just conclude. Yeah, sure. I would just note that the City Council tasked this administration to build the Office of Algorithmic Accountability by June of this last year.

And so once again, New Yorkers should not have to endure what our elected officials have the power to prevent. Thank you so much.

Thank you. Mr. Chaudhry Hassan Mahmood, you may begin. Thank you.

Good morning, Chair and members of the committee. My name is Chaudhry Mahmood. I am a permanent employee of Fire Department Bureau of Enterprise Technology at 12 years of service. Um, on October 8th, the council held a landmark hearing on AI risk. I am here to tell you the risk that lies inside the city agency right now, not in the theory but not in the practice.

In FDNY, BET, artificial intelligence tools are being deployed by the private contractors on emergency dispatch system call centers with PAII and PHI involved. Both the candidate aided dispatch system, which is FireCAD, EMSCAD, with no AI governance policy, no civil servant review or public disclosure. These are the systems that route fire engines, ambulances. They contain 911 call data, EMS patient medical records, and fire marshal investigative information.

So I know there is, uh, because, uh, there is, uh, by the 5th of October, third-party validation, workspace impact reporting, whistleblower protection, all of them apply directly to what's happening to everyone right now. I am asking the committee to require, uh, city agency to publish a public AI tool registry, FOIL-able and controller-audited, mandated civil service sign-off before any AI tool being used with PII, PHI, or public safety data.

They extend the whistleblower protection, as Kim also said, uh, explicitly in AI governance disclosure. So I am— I have written full testimony which I'm going to submit after this. I am available to brief staff confidentially if it is required. The risk is not coming for sure, it is already inside the building. Thank you.

Thank you. I want to acknowledge we've been joined also by Councilmember Wong. Thank you for being here. If we have inadvertently missed anyone who has registered to testify today and has yet to be called on, please use the Zoom hand raise function, and you will be called on in the order your hand is raised. Okay. Thank you so much, Chair De La Rosa.

I really want to say that the answers that we heard from SpaceX AI today were wholly and completely unsatisfactory. The idea that the basically company line about constant focus on safety was really belied by several things. First of all, the fact that the very compelling whistleblower testimony we heard from Devin Kim, whose testimony we were able to secure early this morning, and we appreciate him being here and his forthrightness But the idea that SpaceX AI could not comment or couldn't about anything around his termination was extremely troubling at best.

Secondly, as we discussed, we know that for thousands and thousands of women and children who have had hypersexualized images of them posted online repeatedly, and then hearing from Mr. Kim, who was one of SpaceX AI's first employees, that the company thought that tasteful nudes were appropriate is really shocking, offensive, and wholly and completely unacceptable. And the fact that within seconds I could pull up a hypersexualized image of myself, numerous images, and in fact, not only could we find them, But Rock even had a way where you could make them, like, more sexualized.

You can make it sort of pump it up even more and more and more. This is not at all in line with the testimony we heard from SpaceX AI that they're focused on safety. So I was very deeply disappointed and found their testimony not credible and unacceptable. I— we were very pleased to be joined by Mr. Kim, and we thank him for his bravery and courage in coming forward. and everyone else who testified today.

I'll pass it over to the chair.

Thank you, Speaker Menin, for your leadership, and to all the committee members for being here. This is not the end of the conversation. This council will continue to move legislation on this issue, continue to speak about the Office of Algorithmic Accountability and other issues that are relevant to this topic.

We thank everyone who has joined us today for testimonies, and this hearing is adjourned.