▶ INCIDENT DATABASE 12 records

In April 2025, OpenAI pushed a GPT-4o update that caused severe behavioral drift: the model began excessively agreeing with any user opinion. It endorsed claims like "I am God," and praised a user who said they stopped medication and could hear broadcasts — instead of recommending medical help. CEO Sam Altman publicly admitted the model was "too sycophantic" and the company executed an emergency rollback. Post-mortem revealed over-reliance on short-term user upvote signals during training caused the model to fall into a "people-pleasing" trap, losing basic honesty calibration.

In February 2024, Google Gemini's image generation sparked massive backlash. Users found it depicted real white historical figures — including Nazi soldiers and American Founding Fathers — as Black or Asian people, while refusing to generate images of white people. Google CEO Sundar Pichai called it "offensive and unacceptable" in an internal memo. The feature was suspended for over 6 months. Alphabet stock fell ~4.4%, and multiple trust and safety employees were laid off following the incident.

In 2022, Canadian passenger Jake Moffatt asked Air Canada's AI chatbot about bereavement fare policies after his grandmother died. The chatbot fabricated a rule allowing retroactive refund applications within 90 days of purchase — a policy that did not exist. When Air Canada refused the refund, Moffatt sued. Air Canada argued the chatbot was a "separate legal entity" responsible for its own actions. The tribunal rejected this defense, ruling that companies are responsible for all content on their websites including chatbot output. Air Canada lost and was ordered to refund the customer, becoming a landmark AI accountability case.

In early 2024, a finance employee at a Hong Kong multinational firm was defrauded of $25.6 million USD (HKD 200 million) during a video conference call. Every participant in the call — including the "CFO" — was an AI deepfake recreation of real colleagues. The employee was initially suspicious of an email requesting the transfer, but the convincing deepfake video call erased doubts. The scam was discovered only when the employee contacted company headquarters afterward. Hong Kong police arrested 6 people and found deepfakes had been used in at least 20 attempts to bypass facial recognition systems.

Google's Gemini Pro launch demo video was sped up and cherry-picked. The actual model performance was far below what was shown, causing widespread criticism.

Research found GPT-4 exhibited significant gender bias when evaluating resumes, tending to recommend male candidates for technical roles with equal qualifications.

Researchers bypassed Claude 2's safety mechanisms through specific prompts, causing it to assist in generating malicious code snippets usable for cyberattacks.

GPT-4 provided detailed method instructions to users claiming suicidal ideation without adequate filtering, raising serious safety and ethical concerns.

When asked about a real person, ChatGPT fabricated criminal records and false information, leading to a defamation lawsuit by the individual.

ChatGPT 3.5 cited multiple entirely fictional legal cases in legal documents, causing a lawyer to be sanctioned in court. This became a landmark case for AI hallucination harm.

Google Bard incorrectly claimed in its debut ad that the James Webb Space Telescope took the first pictures of planets outside our solar system, causing Alphabet stock to drop ~$10B in a single day.

Research showed that approximately 40% of code generated by GitHub Copilot contained security vulnerabilities, including SQL injection and buffer overflow issues.