Bixby ( ) is a virtual assistant developed by Samsung Electronics that runs on various Samsung-branded appliances, primarily mobile devices but also some refrigerators televisions and PCs. It uses voice commands and a natural-language user interface to answer questions and perform tasks, while adapting to the users' preferences and behavior. Samsung first launched Bixby in 2017. Along with Bixby voice assistant, its other main component currently is Bixby Vision, a contextual and visual search augmented reality camera app. Formerly, the Bixby suite consisted of a number of other tools, but these have since been renamed, such as Bixby Routines (now Modes and Routines). == History == On 20 March 2017, Samsung announced the voice-powered digital assistant named "Bixby" as a replacement of the S Voice assistant. It was introduced alongside the Galaxy S8 and S8+ and the Galaxy Tab A (2017) during the Galaxy Unpacked 2017 event. Although released for these devices, it could also be sideloaded on older Galaxy devices running Android Nougat. Before the phone's release, the Bixby Button was reprogrammable and could be set to open other applications or assistants, such as Google Assistant. However, near the phone's release, this ability was removed with a firmware update. Remapping remained possible through third-party apps. Bixby was launched in Korean on 1 May 2017 (KST). Bixby Voice was intended to be made available in the US later that spring. However, Samsung postponed the release, as Bixby had issues understanding English. The English version was finally rolled out in July 2017, followed by a Chinese language version later that year. In October 2017, Samsung announced the release of Bixby 2.0 during its annual developer conference in San Francisco. The new version was rolled out across the company's line of connected products, including smartphones, TVs, and refrigerators. Third parties were allowed to develop applications for Bixby using the Samsung Developer Kit. In August 2018, Samsung announced the Bixby-integrated Galaxy Home smart speaker. In 2019, UX developers at Samsung stated that they intended to use AR Emoji avatars as a personified Bixby assistant. At SDC19, Samsung displayed the Galaxy Home Mini speaker, which also supported Bixby. Bixby 3.0 was released with One UI 3 at the start of 2021. With version 3.0, Home and Reminders features were separated from Bixby. In June 2021, screenshots surfaced for what some thought as a replacement for Bixby. The three-dimensional virtual assistant, Sam, was popular on social media, though it was not intended as a replacement for Bixby. Bixby launched for Microsoft Windows in October 2021, with distribution through the Microsoft Store. This version of Bixby was optimized for Samsung's Galaxy Book computers. Samsung launched an AI Bixby custom voice creator in 2023, allowing users to record their own voice commands. Most recently, in July 2024, Samsung confirmed that it plans to launch an upgraded version of Bixby later that year. This new Bixby would be powered by Samsung's proprietary large language model (LLM) technology, promising a significant boost to Bixby's capabilities with the help of generative AI. In January 2025, with the announcement of Galaxy S25 and the One UI 7 update, Bixby was no longer the default voice assistant, having been replaced by Google Gemini. Despite this, Bixby still continued to be developed and expanded by Samsung and was revamped at the same time with new AI capabilities. Samsung brought the "smarter" Bixby to Samsung televisions, allowing users to speak to their TV sets and control their homes with it. A visual refresh was planned for One UI 8.5. == Functionality == Bixby is a voice assistant developed by Samsung that provides device control, information retrieval, and task automation using voice input and artificial intelligence. It can answer contextual queries, adjust system settings, perform searches, and manage reminders or schedules. The service also personalizes responses by recognizing individual user voices. Bixby itself was also formerly called Bixby Voice to differentiate from other Bixby tools in the suite. === Bixby Vision === Bixby Vision is a visual recognition feature that analyzes images captured through the device camera and provides context-specific information or actions. It combines on-device processing with cloud-based AI resources to identify objects, detect text, and interpret scenes within supported applications. It comes pre-installed on Samsung Galaxy phones. It is considered to be the imaging component of Bixby. ==== Translate ==== Detects foreign text in the camera view and provides real-time translation by overlaying translated text on the preview. ==== Text ==== Uses optical character recognition(OCR) to extract printed or handwritten text for copying, searching, or sharing. ==== Discover ==== Identifies consumer products, fashion items, or furniture and retrieves visually similar items or related online information. ==== Wine ==== Recognizes wine labels and provides information such as variety, region of origin, average price, and reviews. ==== Scene Describer ==== Generates written and spoken descriptions of captured scenes, supporting accessibility for users with visual impairments. ==== Object Identifier ==== Identifies plants, animals, food items, or landmarks and displays corresponding names or classification details. ==== Text Reader ==== Converts detected text into spoken audio using text-to-speech functionality. ==== Color Detector ==== Identifies and names colors within the frame, displaying or reading the recognized color aloud. === Former Bixby tools === Bixby Home was a vertically scrolling home screen displaying cards of information such as weather, fitness activity, and smart home controls. It was renamed Samsung Daily with the release of One UI 2.1 in 2020, then replaced by Samsung Free in One UI 3.0. Samsung Free was eventually discontinued in some markets. Its successor, Samsung News, now functions as a news aggregation service with optional home-screen integration similar to Bixby Home. Bixby Routines was an automation feature that allowed users to create custom rules based on triggers such as time, location, or device conditions. Beginning with One UI 5.0, it was renamed Modes and Routines. Bixby Text Call, introduced in One UI 5.0 (2022) in select regions, enabled users to handle incoming calls via speech-to-text conversion and vice versa. It is now named simply Text Call and can be found in the Phone app settings. Bixby Touch allowed users to trigger context-aware actions by touching on-screen content. It analyzed images, text, and other visual elements displayed on the device and provided related options such as translation, image search, product lookup, or other content-based information. Several of its capabilities overlapped with, or were later superseded by, features offered through Bixby Vision. Other legacy components including Bixby Touch, Bixby Global Action, Bixby Dictation, and Bixby Wakeup, formed part of the early Bixby suite and have since been phased out, though exact discontinuation details vary by region. == Regions and languages == As of April 2018, Bixby is available in over 195 countries, but only in Korean, English (American), and Chinese (Mandarin). The limitation is that the models not intended for the Japanese market, like S10e, are not allowed to login to Bixby services from Japan; therefore Bixby becomes blocked. The choice of languages has since expanded: Samsung has deployed Bixby's voice command function in French, and on 20 February 2019 Samsung announced the addition of further languages: English (British), German, Italian and Spanish (Spain). On 22 February 2020, Samsung announced the addition of Portuguese (Brazil), for Galaxy S10 & Note10, in Beta, and later for other models. == Compatible devices == === Flagship series === Galaxy S series: All models since Galaxy S7 Galaxy Tab S: All models since Galaxy Tab S4 Galaxy Note: All models since Galaxy Note FE and Galaxy Note 8 Galaxy Z series: All models === Other series === Galaxy A Galaxy A6/A6+ (Bixby Home, Reminder and Vision) Galaxy A7 (2017) (available to users in South Korea only; Bixby Home and Reminder only) Galaxy A7 (2018) (Bixby Home, Reminder and Vision only) Galaxy A8 (2018) (including A8 Star; Bixby Home, Reminder and Vision only; S Voice used instead) Galaxy A8s (Bixby Home, Reminder and Vision only) Galaxy A9 (2018)/A9s/A9 Star Pro (including A9 Star and A9 Star Lite; Bixby Home, Reminder and Vision only; S Voice used instead) Galaxy A9 Pro (2019) (Bixby Home, Reminder and Vision only) Galaxy A20 (Bixby Home and Service) Galaxy A21s Galaxy A30s (Bixby Home, Vision, Reminder and Routines) Galaxy A40 (Bixby Home and Reminder) Galaxy A41 (Bixby Home, Vision, Routines and Reminder) Galaxy A50 (Bixby Home, Voice, Vision, Reminder and Routines) Galaxy A50s (Bixby Home, Voice, Vision, Reminder and Routines) G
Thermal attack
A thermal attack (aka thermal imaging attack) is an approach that exploits heat traces to uncover the entered credentials. These attacks rely on the phenomenon of heat transfer from one object to another. During authentication, heat transfers from the users' hands to the surface they are interacting with, leaving heat traces behind that can be analyzed using thermal cameras that operate in the far-infrared spectrum. These traces can be recovered and used to reconstruct the passwords. In some cases, the attack can be successful even 30 seconds after the user has authenticated. Thermal attacks can be performed after the victim had authenticated, alleviating the need for in-situ observation attacks (e.g., shoulder surfing attacks) that can be affected by hand occlusions. While smudge attacks can reveal the order of entries of graphical passwords, such as the Android Lock Patterns, thermal attacks can reveal the order of entries even in the case of PINs or alphanumeric passwords. The reason thermal attacks leak information about the order of entry is because keys and buttons that the user touches first lose heat over time, while recently touched ones maintain the heat signature for a longer time. This results in distinguishable heat patterns that can tell the attacker which entry was entered first. Thermal attacks were shown to be effective against plastic keypads, such as the ones used to enter credit card's PINs in supermarkets and restaurants, and on handheld mobile devices such as smartphones and tablets. In their paper published at the Conference on Human Factors in Computing Systems (CHI 2017), Abdelrahman et al. showed that the attack is feasible on today's smartphones. They also proposed some ways to mitigate the attack, such as swiping randomly on the screen to distort the heat traces, or forcing maximum CPU usage for a few seconds. Thermal attacks can also infer passwords from heat traces on keyboards. Researchers at the University of Glasgow showed that attackers who use AI methods can be more effective in performing thermal attacks. Their study presents a new tool called ThermoSecure and evaluates it in two user studies. The results show that ThermoSecure can successfully attack passwords with an average accuracy of 92% to 55%, depending on the length of the password. The effectiveness of thermal attacks also depends on typing behavior and the material of the keycaps. ABS keycaps, which retain heat traces longer, are more vulnerable to thermal attacks. The study also discusses ways to protect against thermal attacks and presents seven potential mitigation approaches. Dr Khamis, who led the development of the technology with Norah Alotaibi and John Williamson, said with thermal imaging cameras more affordable than ever and machine learning becoming more accessible, it was "very likely that people around the world are developing systems along similar lines to ThermoSecure in order to steal passwords". == Thermal Attack Mitigation == === Simple and Practical Measures === One basic and effective way to mitigate thermal attacks is to deliberately create heat noise over the input interface, such as a keypad or keyboard, after entering a password. For instance, placing one's palm over the entire interface for a few seconds after use can obscure the thermal pattern left by the fingers, making it much more difficult for an unauthorized user to interpret the heat traces. === Range of Proposed Strategies === In addition to simple methods, researchers have developed a spectrum of mitigation strategies to counter thermal attacks. These strategies encompass 15 different approaches including: Use of Biometrics: Replacing traditional pin codes or passwords with biometric authentication, such as fingerprint recognition or facial recognition, eliminates the issue of residual heat on keypads. Heating the Interface: Implementing technology to slightly warm up the keypad can effectively neutralize the heat traces left by fingers, preventing thermal cameras from capturing the pattern. Randomizing Key Layouts: Employing dynamic key layouts that change positions every time the interface is used, making it impossible to correlate heat patterns with static input positions. === Technological Intervention on Thermal Cameras === Another avenue for mitigation is to address the issue at the source by modifying thermal cameras. Proposals have been made to develop thermal cameras that can automatically detect vulnerable interfaces such as keyboards or keypads. When these interfaces are detected within the camera's field of view, the camera would be programmed to prevent the user from recording images of them. This solution, however, would require widespread adoption by thermal camera manufacturers. Additionally, the approach is particularly viable for thermal cameras connected to a computing device, such as a smartphone, which can process the images in real time. Many affordable thermal cameras are standalone and do not have connectivity or processing capabilities. However, thermal cameras designed for connection to mobile devices can utilize the smartphone's processing power, making this mitigation approach feasible for such devices.
Rnn (software)
rnn is an open-source machine learning framework that implements recurrent neural network architectures, such as LSTM and GRU, natively in the R programming language, that has been downloaded over 100,000 times (from the RStudio servers alone). The rnn package is distributed through the Comprehensive R Archive Network under the open-source GPL v3 license. == Workflow == The below example from the rnn documentation show how to train a recurrent neural network to solve the problem of bit-by-bit binary addition. == sigmoid == The sigmoid functions and derivatives used in the package were originally included in the package, from version 0.8.0 onwards, these were released in a separate R package sigmoid, with the intention to enable more general use. The sigmoid package is a dependency of the rnn package and therefore automatically installed with it. == Reception == With the release of version 0.3.0 in April 2016 the use in production and research environments became more widespread. The package was reviewed several months later on the R blog The Beginner Programmer as "R provides a simple and very user friendly package named rnn for working with recurrent neural networks.", which further increased usage. The book Neural Networks in R by Balaji Venkateswaran and Giuseppe Ciaburro uses rnn to demonstrate recurrent neural networks to R users. It is also used in the r-exercises.com course "Neural network exercises". The RStudio CRAN mirror download logs show that the package is downloaded on average about 2,000 per month from those servers , with a total of over 100,000 downloads since the first release, according to RDocumentation.org, this puts the package in the 15th percentile of most popular R packages .
KataGo
KataGo is a free and open-source computer Go program, capable of defeating top-level human players. First released on 27 February 2019, it is developed by David Wu, who also developed the Arimaa playing program bot_Sharp which defeated three top human players to win the Arimaa AI Challenge in 2015. KataGo's first release was trained by David Wu using resources provided by his employer Jane Street Capital, but it is now trained by a distributed effort. Members of the computer Go community provide computing resources by running the client, which generates self-play games and rating games, and submits them to a server. The self-play games are used to train newer networks and the rating games to evaluate the networks' relative strengths. KataGo supports the Go Text Protocol, with various extensions, thus making it compatible with popular GUIs such as Lizzie. As an alternative, it also implements a custom "analysis engine" protocol, which is used by the KaTrain GUI, among others. KataGo is widely used by strong human go players, including the South Korean national team, for training purposes. KataGo is also used as the default analysis engine in the online Go website AI Sensei, as well as OGS (the Online Go Server). == Technology == Based on techniques used by DeepMind's AlphaGo Zero, KataGo implements Monte Carlo tree search with a convolutional neural network providing position evaluation and policy guidance. Compared to AlphaGo, KataGo introduces many refinements that enable it to learn faster and play more strongly. Notable features of KataGo that are absent in many other Go-playing programs include score estimation; support for small boards, rectangular boards, and large boards; arbitrary values of komi and handicaps; and the ability to use various Go rulesets and adjust its play and evaluation for the small differences between them. === Network === The network used in KataGo are ResNets with pre-activation. While AlphaGo Zero has only game board history as input features (as it was designed as a general architecture for board games, subsequently becoming AlphaZero), the input to the network contains additional features designed by hand specifically for playing Go. These features include liberties, komi parity, pass-alive, and ladders. The trunk is essentially the same as in AlphaGo Zero, but with global pooling layers added to allow the network to be conditioned on global context such as ko fights. This is similar to the Squeeze-and-Excitation Network. The network has two heads: a policy head and a value head. The policy and value heads are mostly the same as in AlphaGo Zero, but both heads have auxiliary subheads to provide auxiliary loss signal for faster training: Policy head: predicts policy for the current player's move this turn, and the opponent player's move in the next turn. A policy Each is a logit array of size 19 × 19 + 1 {\displaystyle 19\times 19+1} , representing the logit of making a move in one of the points, plus the logit of passing. Value head: predicts game outcome, expected score difference, expected board ownership, etc. The network is described in detail in Appendix A of the report. The code base switched from using TensorFlow to PyTorch in version 1.12. === Training === Let its trunk have b {\displaystyle b} residual blocks and c {\displaystyle c} channels. During its first training run, multiple networks were trained with increasing ( b , c ) {\displaystyle (b,c)} . It took 19 days using a maximum of 28 Nvidia V100 GPUs at 4.2 million games. After the first training run, training became a distributed project run by volunteers, with increasing network sizes. As of August 2024, it has reached b28c512 (28 blocks, 512 channels). == Adversarial attacks == In 2022, KataGo was used as the target for adversarial attack research, designed to demonstrate the "surprising failure modes" of AI systems. The researchers were able to trick KataGo into ending the game prematurely. Adversarial training improves defense against adversarial attacks, though not perfectly.
PauseAI
PauseAI is a global political movement founded in the Netherlands with the stated aim of achieving global coordination to stop the development of more powerful general artificial intelligence systems, at least until it is known how to build them safely, and keep them under democratic control. The movement was established in Utrecht in May 2023 by software entrepreneur Joep Meindertsma. == Proposal == PauseAI's stated goal is to "implement a temporary pause on the training of the most powerful general AI systems". Their website lists some proposed steps to achieve this goal: Set up an international AI safety agency, similar to the IAEA. Only allow training of general AI systems if their safety can be guaranteed. Only allow deployment of models after no dangerous capabilities are present. == Background == During the late 2010s and early 2020s, a rapid improvement in the capabilities of artificial intelligence models known as the AI boom was underway, which included the release of large language model GPT-3, its more powerful successor GPT-4, and image generation models Midjourney and DALL-E. This led to an increased concern about the risks of advanced AI, causing the Future of Life Institute to release an open letter calling for "all AI labs to immediately pause for at least six months the training of AI systems more powerful than GPT-4". The letter was signed by thousands of AI researchers and industry CEOs such as Yoshua Bengio, Stuart Russell, and Elon Musk. == History == Founder Joep Meindertsma first became worried about the existential risk from artificial intelligence after reading philosopher Nick Bostrom's 2014 book Superintelligence: Paths, Dangers, Strategies. He founded PauseAI in May 2023, putting his job as the CEO of a software firm on hold. Meindertsma claimed the rate of progress in AI alignment research is lagging behind the progress in AI capabilities, and said "there is a chance that we are facing extinction in a short frame of time". As such, he felt an urge to organise people to act. PauseAI's first public action was to protest in front of Microsoft's Brussels lobbying office in May 2023 during an event on artificial intelligence. In November of the same year, they protested outside the inaugural AI Safety Summit at Bletchley Park. The Bletchley Declaration that was signed at the summit, which acknowledged the potential for catastrophic risks stemming from AI, was perceived by Meindertsma to be a small first step. But, he argued "binding international treaties" are needed. He mentioned the Montreal Protocol and treaties banning blinding laser weapons as examples of previous successful global agreements. In February 2024, members of PauseAI gathered outside OpenAI's headquarters in San Francisco, in part due to OpenAI changing its usage policy that prohibited the use of its models for military purposes. On 13 May 2024, protests were held across thirteen countries before the AI Seoul Summit, including the United States, the United Kingdom, Brazil, Germany, Australia, and Norway. Meindertserma said that those attending the summit "need to realize that they are the only ones who have the power to stop this race". Protesters in San Francisco held signs reading "When in doubt, pause", and "Quit your job at OpenAI. Trust your conscience". Jan Leike, head of the "superalignment" team at OpenAI, resigned two days later due to his belief that "safety culture and processes [had] taken a backseat to shiny products".
Alexander Y. Tetelbaum
Alexander Y. Tetelbaum (born August 16, 1948) is a Ukrainian American computer scientist, inventor, and academic who has contributed to electronic design automation (EDA) and artificial intelligence (AI) since the late 1960s; and holds 46 U.S. patents in EDA and related fields. Tetelbaum is the founding president of International Solomon University, the first Jewish university in Ukraine, established during a period of renewed efforts to address antisemitism in Ukraine. == Early life and education == He graduated from a Kyiv mathematical high school with a silver medal in 1966. Tetelbaum enrolled at the Kyiv Polytechnic Institute (KPI), now National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute" in 1966, graduating in 1972 with an MS in Electronics with honors. He earned his PhD in Electrical and Computer Engineering from KPI in 1975, with a dissertation on electronic design automation, and his Doctor of Engineering Science in 1986. == Academic career == Tetelbaum began his academic career at KPI in 1973 as a junior scientist, becoming a professor in the Computer and Electrical Engineering Department in 1980. Later, he founded and served as president of International Solomon University in Kyiv from 1991 to 1996, the first Jewish university in Ukraine. The university became a major academic center for computer science and Jewish studies in the post-Soviet era. He was a visiting and adjunct professor at Michigan State University from 1993 to 1996. == Professional career == Tetelbaum worked as an engineer at the Kiev Institute of Cybernetics from 1972 to 1973, and later, he led the Design Automation Lab at Kyiv Polytechnic Institute from 1975 to 1987. In the United States, he served as EDA manager at Silicon Graphics Corporation from 1996 to 1998 and principal engineer at LSI Corporation from 1998 to 2012. He founded and served as CEO of Abelite Design Automation, Inc., from 2012 to 2022. == Contributions in computer science == Tetelbaum has contributed to electronic design automation (EDA) and artificial intelligence (AI) since the 1960s. His early work included methods for EDA, particularly physical design automation and mathematical optimization; and he developed force-directed placement and topological routing methods. Tetelbaum generalized Rent's rule for hierarchical systems and large blocks, proposing a graph-based framework that extends applicability to arbitrary partition sizes with improved accuracy. Additional IEEE and related conference contributions from the mid-1990s include: "Path Search for Complicated Function", 1995 IEEE International Symposium on Circuits and Systems "A Performance-driven Placement Approach of Standard Cells" (International Conference on Intelligent Systems, 1995) "Framework of a New Methodology for Behavioral to Physical Design Linkage" (38th Midwest Symposium on Circuits and Systems, 1996) Statistical timing design and variations Test Methodologies These and other works and patents contributed to timing-driven placement, crosstalk reduction, clock tree synthesis, and interconnect optimization in VLSI design. == Patents == Tetelbaum holds 46 U.S. patents in EDA and related fields. Notable examples include: For the full list of patents, see Justia Patents or Google Patents. == Publications == === Early publications in the Soviet Union === Before the appearance of American books on electronic design automation (EDA), Tetelbaum published several scientific books and monographs on the subject in Russian/Ukrainian. Electronic Design Automation, Kiev: Znanie Publisher, 1975. Planar Design of Electronic Circuits, Kiev: Znanie Publisher, 1977. Formal Design of Computer Systems, Moscow: Sovetskoe Radio, 1979. CAD of Electronic Equipment: Topological Approach, Kiev: Vyssha Shkola, 1980; 2nd ed. 1981. Automated Design of Electronic Circuits (1981) CAD of VLSI Circuits, Kiev: Vyssha Shkola, 1983. Topological Algorithms of Multilayer Printed Circuit Boards Routing, Moscow: Radio i Svyaz, 1983. CAD of VLSI Circuits on Master Slice Chips, Moscow: Radio i Svyaz, 1988. Increasing the Effectiveness of CAD Systems, Kiev: UMKVO, 1991. === Scientific Monographs (English) === Minimum Number of Timing Signoff Corners (2022) Interviewing AI (2026) The AI Debate (2026) New Nostradamus Predictions: 2026: The Next Decade & Beyond (2035–2050+) (2026) For a consolidated record of Tetelbaum's publications, see Alexander Y. Tetelbaum, Wikidata Q4720205. === Other publications === Tetelbaum also published educational books on problem-solving methods: Yes-No Puzzles-Games Puzzle Games for Kids Solving Non-Standard Problems Solving Non-Standard Very Hard Problems Additionally, Tetelbaum published three thrillers: Omerta Operations Executive Director Eruption Yacht Finally, he published his memoir and an entertaining book: Unfinished Equations Artificially Intelligent Humor
Capsule neural network
A capsule neural network (CapsNet) is a machine learning system that is a type of artificial neural network (ANN) that can be used to better model hierarchical relationships. The approach is an attempt to more closely mimic biological neural organization. The idea is to add structures called "capsules" to a convolutional neural network (CNN), and to reuse output from several of those capsules to form more stable (with respect to various perturbations) representations for higher capsules. The output is a vector consisting of the probability of an observation, and a pose for that observation. This vector is similar to what is done for example when doing classification with localization in CNNs. Among other benefits, capsnets address the "Picasso problem" in image recognition: images that have all the right parts but that are not in the correct spatial relationship (e.g., in a "face", the positions of the mouth and one eye are switched). For image recognition, capsnets exploit the fact that while viewpoint changes have nonlinear effects at the pixel level, they have linear effects at the part/object level. This can be compared to inverting the rendering of an object of multiple parts. == History == In 2000, Geoffrey Hinton et al. described an imaging system that combined segmentation and recognition into a single inference process using parse trees. So-called credibility networks described the joint distribution over the latent variables and over the possible parse trees. That system proved useful on the MNIST handwritten digit database. A dynamic routing mechanism for capsule networks was introduced by Hinton and his team in 2017. The approach was claimed to reduce error rates on MNIST and to reduce training set sizes. Results were claimed to be considerably better than a CNN on highly overlapped digits. In Hinton's original idea one minicolumn would represent and detect one multidimensional entity. == Transformations == An invariant is an object property that does not change as a result of some transformation. For example, the area of a circle does not change if the circle is shifted to the left. Informally, an equivariant is a property that changes predictably under transformation. For example, the center of a circle moves by the same amount as the circle when shifted. A nonequivariant is a property whose value does not change predictably under a transformation. For example, transforming a circle into an ellipse means that its perimeter can no longer be computed as π times the diameter. In computer vision, the class of an object is expected to be an invariant over many transformations. I.e., a cat is still a cat if it is shifted, turned upside down or shrunken in size. However, many other properties are instead equivariant. The volume of a cat changes when it is scaled. Equivariant properties such as a spatial relationship are captured in a pose, data that describes an object's translation, rotation, scale and reflection. Translation is a change in location in one or more dimensions. Rotation is a change in orientation. Scale is a change in size. Reflection is a mirror image. Unsupervised capsnets learn a global linear manifold between an object and its pose as a matrix of weights. In other words, capsnets can identify an object independent of its pose, rather than having to learn to recognize the object while including its spatial relationships as part of the object. In capsnets, the pose can incorporate properties other than spatial relationships, e.g., color (cats can be of various colors). Multiplying the object by the manifold poses the object (for an object, in space). == Pooling == Capsnets reject the pooling layer strategy of conventional CNNs that reduces the amount of detail to be processed at the next higher layer. Pooling allows a degree of translational invariance (it can recognize the same object in a somewhat different location) and allows a larger number of feature types to be represented. Capsnet proponents argue that pooling: violates biological shape perception in that it has no intrinsic coordinate frame; provides invariance (discarding positional information) instead of equivariance (disentangling that information); ignores the linear manifold that underlies many variations among images; routes statically instead of communicating a potential "find" to the feature that can appreciate it; damages nearby feature detectors, by deleting the information they rely upon. == Capsules == A capsule is a set of neurons that individually activate for various properties of a type of object, such as position, size and hue. Formally, a capsule is a set of neurons that collectively produce an activity vector with one element for each neuron to hold that neuron's instantiation value (e.g., hue). Graphics programs use instantiation value to draw an object. Capsnets attempt to derive these from their input. The probability of the entity's presence in a specific input is the vector's length, while the vector's orientation quantifies the capsule's properties. Artificial neurons traditionally output a scalar, real-valued activation that loosely represents the probability of an observation. Capsnets replace scalar-output feature detectors with vector-output capsules and max-pooling with routing-by-agreement. Because capsules are independent, when multiple capsules agree, the probability of correct detection is much higher. A minimal cluster of two capsules considering a six-dimensional entity would agree within 10% by chance only once in a million trials. As the number of dimensions increase, the likelihood of a chance agreement across a larger cluster with higher dimensions decreases exponentially. Capsules in higher layers take outputs from capsules at lower layers, and accept those whose outputs cluster. A cluster causes the higher capsule to output a high probability of observation that an entity is present and also output a high-dimensional (20-50+) pose. Higher-level capsules ignore outliers, concentrating on clusters. This is similar to the Hough transform, the RHT and RANSAC from classic digital image processing. == Routing by agreement == The outputs from one capsule (child) are routed to capsules in the next layer (parent) according to the child's ability to predict the parents' outputs. Over the course of a few iterations, each parents' outputs may converge with the predictions of some children and diverge from those of others, meaning that that parent is present or absent from the scene. For each possible parent, each child computes a prediction vector by multiplying its output by a weight matrix (trained by backpropagation). Next the output of the parent is computed as the scalar product of a prediction with a coefficient representing the probability that this child belongs to that parent. A child whose predictions are relatively close to the resulting output successively increases the coefficient between that parent and child and decreases it for parents that it matches less well. This increases the contribution that that child makes to that parent, thus increasing the scalar product of the capsule's prediction with the parent's output. After a few iterations, the coefficients strongly connect a parent to its most likely children, indicating that the presence of the children imply the presence of the parent in the scene. The more children whose predictions are close to a parent's output, the more quickly the coefficients grow, driving convergence. The pose of the parent (reflected in its output) progressively becomes compatible with that of its children. The coefficients' initial logits are the log prior probabilities that a child belongs to a parent. The priors can be trained discriminatively along with the weights. The priors depend on the location and type of the child and parent capsules, but not on the current input. At each iteration, the coefficients are adjusted via a "routing" softmax so that they continue to sum to 1 (to express the probability that a given capsule is the parent of a given child.) Softmax amplifies larger values and diminishes smaller values beyond their proportion of the total. Similarly, the probability that a feature is present in the input is exaggerated by a nonlinear "squashing" function that reduces values (smaller ones drastically and larger ones such that they are less than 1). This dynamic routing mechanism provides the necessary deprecation of alternatives ("explaining away") that is needed for segmenting overlapped objects. This learned routing of signals has no clear biological equivalent. Some operations can be found in cortical layers, but they do not seem to relate this technique. === Math/code === The pose vector u i {\textstyle \mathbf {u} _{i}} is rotated and translated by a matrix W i j {\textstyle \mathbf {W} _{ij}} into a vector u ^ j | i {\textstyle \mathbf {\hat {u}} _{j|i}} that predicts the output of the parent capsule. u ^ j | i = W i j u i {\displaystyle \mathbf {