Explore together how to cheat, or “spoof”, a machine vision system.
Keywords: Classifier, eXplainable AI, machine vision
Breakable Machine is a browser-based learning game for the whole class, in which students do not train AI systems but break them. The tool allows learners to explore how machine vision -based AI systems can be misled by small changes in the environment, appearance, or camera angle.
Learners try to trick an AI system into misclassifying an image. In doing so, they discover that AI systems can be fragile, easily misled, and strongly dependent on the data that were used to train them.
Breakable Machine makes AI behavior visible through eXplainable AI (XAI).
A heatmap view highlights which parts of an image most influence the classification result and why seemingly irrelevant features, such as background, lighting, or a single object, can strongly influence the outcome. Learners see that high classification confidence does not necessarily mean a correct result.
The game is collaborative: observations are shared, strategies are compared, and learners collectively reflect on why the system behaves the way it does. Breakable Machine supports critical AI literacy by helping learners understand the limitations, risks, and societal implications of AI systems.
The tool runs entirely in the browser, locally in the classroom, and does not collect or store personal data.
Key concepts: Salient features, confidence vs. correctness, generalization, misclassification, model limitations, spoofing, brittleness, responsibility for undesirable outcomes
Open beta available since 2025. Estimated public release: Spring 2026.