In the areas of computer vision, image analysis and signal processing, the notion of scale-space representation is used for processing measurement data at multiple scales, and specifically enhance or suppress image features over different ranges of scale (see the article on scale space). A special type of scale-space representation is provided by the Gaussian scale space, where the image data in N dimensions is subjected to smoothing by Gaussian convolution. Most of the theory for Gaussian scale space deals with continuous images, whereas one when implementing this theory will have to face the fact that most measurement data are discrete. Hence, the theoretical problem arises concerning how to discretize the continuous theory while either preserving or well approximating the desirable theoretical properties that lead to the choice of the Gaussian kernel (see the article on scale-space axioms). This article describes basic approaches for this that have been developed in the literature, see also for an in-depth treatment regarding the topic of approximating the Gaussian smoothing operation and the Gaussian derivative computations in scale-space theory, and for a complementary treatment regarding hybrid discretization methods. == Statement of the problem == The Gaussian scale-space representation of an N-dimensional continuous signal, f C ( x 1 , ⋯ , x N , t ) , {\displaystyle f_{C}\left(x_{1},\cdots ,x_{N},t\right),} is obtained by convolving fC with an N-dimensional Gaussian kernel: g N ( x 1 , ⋯ , x N , t ) . {\displaystyle g_{N}\left(x_{1},\cdots ,x_{N},t\right).} In other words: L ( x 1 , ⋯ , x N , t ) = ∫ u 1 = − ∞ ∞ ⋯ ∫ u N = − ∞ ∞ f C ( x 1 − u 1 , ⋯ , x N − u N , t ) ⋅ g N ( u 1 , ⋯ , u N , t ) d u 1 ⋯ d u N . {\displaystyle L\left(x_{1},\cdots ,x_{N},t\right)=\int _{u_{1}=-\infty }^{\infty }\cdots \int _{u_{N}=-\infty }^{\infty }f_{C}\left(x_{1}-u_{1},\cdots ,x_{N}-u_{N},t\right)\cdot g_{N}\left(u_{1},\cdots ,u_{N},t\right)\,du_{1}\cdots du_{N}.} However, for implementation, this definition is impractical, since it is continuous. When applying the scale space concept to a discrete signal fD, different approaches can be taken. This article is a brief summary of some of the most frequently used methods. == Separability == Using the separability property of the Gaussian kernel g N ( x 1 , … , x N , t ) = G ( x 1 , t ) ⋯ G ( x N , t ) {\displaystyle g_{N}\left(x_{1},\dots ,x_{N},t\right)=G\left(x_{1},t\right)\cdots G\left(x_{N},t\right)} the N-dimensional convolution operation can be decomposed into a set of separable smoothing steps with a one-dimensional Gaussian kernel G along each dimension L ( x 1 , ⋯ , x N , t ) = ∫ u 1 = − ∞ ∞ ⋯ ∫ u N = − ∞ ∞ f C ( x 1 − u 1 , ⋯ , x N − u N , t ) G ( u 1 , t ) d u 1 ⋯ G ( u N , t ) d u N , {\displaystyle L(x_{1},\cdots ,x_{N},t)=\int _{u_{1}=-\infty }^{\infty }\cdots \int _{u_{N}=-\infty }^{\infty }f_{C}(x_{1}-u_{1},\cdots ,x_{N}-u_{N},t)G(u_{1},t)\,du_{1}\cdots G(u_{N},t)\,du_{N},} where G ( x , t ) = 1 2 π t e − x 2 2 t {\displaystyle G(x,t)={\frac {1}{\sqrt {2\pi t}}}e^{-{\frac {x^{2}}{2t}}}} and the standard deviation of the Gaussian σ is related to the scale parameter t according to t = σ2. Separability will be assumed in all that follows, even when the kernel is not exactly Gaussian, since separation of the dimensions is the most practical way to implement multidimensional smoothing, especially at larger scales. Therefore, the rest of the article focuses on the one-dimensional case. == The sampled Gaussian kernel == When implementing the one-dimensional smoothing step in practice, the presumably simplest approach is to convolve the discrete signal fD with a sampled Gaussian kernel: L ( x , t ) = ∑ n = − ∞ ∞ f ( x − n ) G ( n , t ) {\displaystyle L(x,t)=\sum _{n=-\infty }^{\infty }f(x-n)\,G(n,t)} where G ( n , t ) = 1 2 π t e − n 2 2 t {\displaystyle G(n,t)={\frac {1}{\sqrt {2\pi t}}}e^{-{\frac {n^{2}}{2t}}}} (with t = σ2) which in turn is truncated at the ends to give a filter with finite impulse response L ( x , t ) = ∑ n = − M M f ( x − n ) G ( n , t ) {\displaystyle L(x,t)=\sum _{n=-M}^{M}f(x-n)\,G(n,t)} for M chosen sufficiently large (see error function) such that 2 ∫ M ∞ G ( u , t ) d u = 2 ∫ M t ∞ G ( v , 1 ) d v < ε . {\displaystyle 2\int _{M}^{\infty }G(u,t)\,du=2\int _{\frac {M}{\sqrt {t}}}^{\infty }G(v,1)\,dv<\varepsilon .} A common choice is to set M to a constant C times the standard deviation of the Gaussian kernel M = C σ + 1 = C t + 1 {\displaystyle M=C\sigma +1=C{\sqrt {t}}+1} where C is often chosen somewhere between 3 and 6. Using the sampled Gaussian kernel can, however, lead to implementation problems, in particular when computing higher-order derivatives at finer scales by applying sampled derivatives of Gaussian kernels. When accuracy and robustness are primary design criteria, alternative implementation approaches should therefore be considered. For small values of ε (10−6 to 10−8) the errors introduced by truncating the Gaussian are usually negligible. For larger values of ε, however, there are many better alternatives to a rectangular window function. For example, for a given number of points, a Hamming window, Blackman window, or Kaiser window will do less damage to the spectral and other properties of the Gaussian than a simple truncation will. Notwithstanding this, since the Gaussian kernel decreases rapidly at the tails, the main recommendation is still to use a sufficiently small value of ε such that the truncation effects are no longer important. == The discrete Gaussian kernel == A more refined approach is to convolve the original signal with the discrete Gaussian kernel T(n, t) L ( x , t ) = ∑ n = − ∞ ∞ f ( x − n ) T ( n , t ) {\displaystyle L(x,t)=\sum _{n=-\infty }^{\infty }f(x-n)\,T(n,t)} where T ( n , t ) = e − t I n ( t ) {\displaystyle T(n,t)=e^{-t}I_{n}(t)} and I n ( t ) {\displaystyle I_{n}(t)} denotes the modified Bessel functions of integer order, n. This is the discrete counterpart of the continuous Gaussian in that it is the solution to the discrete diffusion equation (discrete space, continuous time), just as the continuous Gaussian is the solution to the continuous diffusion equation. This filter can be truncated in the spatial domain as for the sampled Gaussian L ( x , t ) = ∑ n = − M M f ( x − n ) T ( n , t ) {\displaystyle L(x,t)=\sum _{n=-M}^{M}f(x-n)\,T(n,t)} or can be implemented in the Fourier domain using a closed-form expression for its discrete-time Fourier transform: T ^ ( θ , t ) = ∑ n = − ∞ ∞ T ( n , t ) e − i θ n = e t ( cos θ − 1 ) . {\displaystyle {\widehat {T}}(\theta ,t)=\sum _{n=-\infty }^{\infty }T(n,t)\,e^{-i\theta n}=e^{t(\cos \theta -1)}.} With this frequency-domain approach, the scale-space properties transfer exactly to the discrete domain, or with excellent approximation using periodic extension and a suitably long discrete Fourier transform to approximate the discrete-time Fourier transform of the signal being smoothed. Moreover, higher-order derivative approximations can be computed in a straightforward manner (and preserving scale-space properties) by applying small support central difference operators to the discrete scale space representation. As with the sampled Gaussian, a plain truncation of the infinite impulse response will in most cases be a sufficient approximation for small values of ε, while for larger values of ε it is better to use either a decomposition of the discrete Gaussian into a cascade of generalized binomial filters or alternatively to construct a finite approximate kernel by multiplying by a window function. If ε has been chosen too large such that effects of the truncation error begin to appear (for example as spurious extrema or spurious responses to higher-order derivative operators), then the options are to decrease the value of ε such that a larger finite kernel is used, with cutoff where the support is very small, or to use a tapered window. == Recursive filters == Since computational efficiency is often important, low-order recursive filters are often used for scale-space smoothing. For example, Young and van Vliet use a third-order recursive filter with one real pole and a pair of complex poles, applied forward and backward to make a sixth-order symmetric approximation to the Gaussian with low computational complexity for any smoothing scale. By relaxing a few of the axioms, Lindeberg concluded that good smoothing filters would be "normalized Pólya frequency sequences", a family of discrete kernels that includes all filters with real poles at 0 < Z < 1 and/or Z > 1, as well as with real zeros at Z < 0. For symmetry, which leads to approximate directional homogeneity, these filters must be further restricted to pairs of poles and zeros that lead to zero-phase filters. To match the transfer function curvature at zero frequency of the discrete Gaussian, which ensures an approximate semi-group property of additive t, two poles at Z = 1 + 2 t − ( 1 + 2 t ) 2 − 1 {\displaystyle
Pixel-art scaling algorithms
Pixel art scaling algorithms are graphical filters that attempt to enhance the appearance of hand-drawn 2D pixel art graphics. These algorithms are a form of automatic image enhancement. Pixel art scaling algorithms employ methods significantly different than the common methods of image rescaling, which have the goal of preserving the appearance of images. As pixel art graphics are commonly used at very low resolutions, they employ careful coloring of individual pixels. This results in graphics that rely on a high amount of stylized visual cues to define complex shapes. Several specialized algorithms have been developed to handle re-scaling of such graphics. These specialized algorithms can improve the appearance of pixel-art graphics, but in doing so they introduce changes. Such changes may be undesirable, especially if the goal is to faithfully reproduce the original appearance. Since a typical application of this technology is improving the appearance of fourth-generation and earlier video games on arcade and console emulators, many pixel art scaling algorithms are designed to run in real-time for sufficiently small input images at 60-frames per second. This places constraints on the type of programming techniques that can be used for this sort of real-time processing. Many work only on specific scale factors. 2× is the most common scale factor, while 3×, 4×, 5×, and 6× exist but are less used. == Algorithms == === SAA5050 'Diagonal Smoothing' === The Mullard SAA5050 Teletext character generator chip (1980) used a primitive pixel scaling algorithm to generate higher-resolution characters on the screen from a lower-resolution representation from its internal ROM. Internally, each character shape was defined on a 5 × 9 pixel grid, which was then interpolated by smoothing diagonals to give a 10 × 18 pixel character, with a characteristically angular shape, surrounded to the top and the left by two pixels of blank space. The algorithm only works on monochrome source data, and assumes the source pixels will be logically true or false depending on whether they are 'on' or 'off'. Pixels 'outside the grid pattern' are assumed to be off. The algorithm works as follows: A B C --\ 1 2 D E F --/ 3 4 1 = B | (A & E & !B & !D) 2 = B | (C & E & !B & !F) 3 = E | (!A & !E & B & D) 4 = E | (!C & !E & B & F) Note that this algorithm, like the Eagle algorithm below, has a flaw: If a pattern of 4 pixels in a hollow diamond shape appears, the hollow will be obliterated by the expansion. The SAA5050's internal character ROM carefully avoids ever using this pattern. The degenerate case: becomes: === EPX/Scale2×/AdvMAME2× === Eric's Pixel Expansion (EPX) is an algorithm developed by Eric Johnston at LucasArts around 1992, when porting the SCUMM engine games from the IBM PC (which ran at 320 × 200 × 256 colors) to the early color Macintosh computers, which ran at more or less double that resolution. The algorithm works as follows, expanding P into 4 new pixels based on P's surroundings: 1=P; 2=P; 3=P; 4=P; IF C==A => 1=A IF A==B => 2=B IF D==C => 3=C IF B==D => 4=D IF of A, B, C, D, three or more are identical: 1=2=3=4=P Later implementations of this same algorithm (as AdvMAME2× and Scale2×, developed around 2001) are slightly more efficient but functionally identical: 1=P; 2=P; 3=P; 4=P; IF C==A AND C!=D AND A!=B => 1=A IF A==B AND A!=C AND B!=D => 2=B IF D==C AND D!=B AND C!=A => 3=C IF B==D AND B!=A AND D!=C => 4=D AdvMAME2× is available in DOSBox via the scaler=advmame2x dosbox.conf option. The AdvMAME4×/Scale4× algorithm is just EPX applied twice to get 4× resolution. ==== Scale3×/AdvMAME3× and ScaleFX ==== The AdvMAME3×/Scale3× algorithm (available in DOSBox via the scaler=advmame3x dosbox.conf option) can be thought of as a generalization of EPX to the 3× case. The corner pixels are calculated identically to EPX. 1=E; 2=E; 3=E; 4=E; 5=E; 6=E; 7=E; 8=E; 9=E; IF D==B AND D!=H AND B!=F => 1=D IF (D==B AND D!=H AND B!=F AND E!=C) OR (B==F AND B!=D AND F!=H AND E!=A) => 2=B IF B==F AND B!=D AND F!=H => 3=F IF (H==D AND H!=F AND D!=B AND E!=A) OR (D==B AND D!=H AND B!=F AND E!=G) => 4=D 5=E IF (B==F AND B!=D AND F!=H AND E!=I) OR (F==H AND F!=B AND H!=D AND E!=C) => 6=F IF H==D AND H!=F AND D!=B => 7=D IF (F==H AND F!=B AND H!=D AND E!=G) OR (H==D AND H!=F AND D!=B AND E!=I) => 8=H IF F==H AND F!=B AND H!=D => 9=F There is also a variant improved over Scale3× called ScaleFX, developed by Sp00kyFox, and a version combined with Reverse-AA called ScaleFX-Hybrid. === Eagle === Eagle works as follows: for every in pixel, we will generate 4 out pixels. First, set all 4 to the color of the pixel we are currently scaling (as nearest-neighbor). Next look at the three pixels above, to the left, and diagonally above left: if all three are the same color as each other, set the top left pixel of our output square to that color in preference to the nearest-neighbor color. Work similarly for all four pixels, and then move to the next one. Assume an input matrix of 3 × 3 pixels where the centermost pixel is the pixel to be scaled, and an output matrix of 2 × 2 pixels (i.e., the scaled pixel) first: |Then . . . --\ CC |S T U --\ 1 2 . C . --/ CC |V C W --/ 3 4 . . . |X Y Z | IF V==S==T => 1=S | IF T==U==W => 2=U | IF V==X==Y => 3=X | IF W==Z==Y => 4=Z Thus if we have a single black pixel on a white background it will vanish. This is a bug in the Eagle algorithm but is solved by other algorithms such as EPX, 2xSaI, and HQ2x. === 2×SaI === 2×SaI, short for 2× Scale and Interpolation engine, was inspired by Eagle. It was designed by Derek Liauw Kie Fa, also known as Kreed, primarily for use in console and computer emulators, and it has remained fairly popular in this niche. Many of the most popular emulators, including ZSNES and VisualBoyAdvance, offer this scaling algorithm as a feature. Several slightly different versions of the scaling algorithm are available, and these are often referred to as Super 2×SaI and Super Eagle. The 2xSaI family works on a 4 × 4 matrix of pixels where the pixel marked A below is scaled: I E F J G A B K --\ W X H C D L --/ Y Z M N O P For 16-bit pixels, they use pixel masks which change based on whether the 16-bit pixel format is 565 or 555. The constants colorMask, lowPixelMask, qColorMask, qLowPixelMask, redBlueMask, and greenMask are 16-bit masks. The lower 8 bits are identical in either pixel format. Two interpolation functions are described: INTERPOLATE(uint32 A, UINT32 B). -- linear midpoint of A and B if (A == B) return A; return ( ((A & colorMask) >> 1) + ((B & colorMask) >> 1) + (A & B & lowPixelMask) ); Q_INTERPOLATE(uint32 A, uint32 B, uint32 C, uint32 D) -- bilinear interpolation; A, B, C, and D's average x = ((A & qColorMask) >> 2) + ((B & qColorMask) >> 2) + ((C & qColorMask) >> 2) + ((D & qColorMask) >> 2); y = (A & qLowPixelMask) + (B & qLowPixelMask) + (C & qLowPixelMask) + (D & qLowPixelMask); y = (y >> 2) & qLowPixelMask; return x + y; The algorithm checks A, B, C, and D for a diagonal match such that A==D and B!=C, or the other way around, or if they are both diagonals or if there is no diagonal match. Within these, it checks for three or four identical pixels. Based on these conditions, the algorithm decides whether to use one of A, B, C, or D, or an interpolation among only these four, for each output pixel. The 2xSaI arbitrary scaler can enlarge any image to any resolution and uses bilinear filtering to interpolate pixels. Since Kreed released the source code under the GNU General Public License, it is freely available to anyone wishing to utilize it in a project released under that license. Developers wishing to use it in a non-GPL project would be required to rewrite the algorithm without using any of Kreed's existing code. It is available in DOSBox via scaler=2xsai option. === hqnx family === Maxim Stepin's hq2x, hq3x, and hq4x are for scale factors of 2:1, 3:1, and 4:1 respectively. Each work by comparing the color value of each pixel to those of its eight immediate neighbors, marking the neighbors as close or distant, and using a pre-generated lookup table to find the proper proportion of input pixels' values for each of the 4, 9 or 16 corresponding output pixels. The hq3x family will perfectly smooth any diagonal line whose slope is ±0.5, ±1, or ±2 and which is not anti-aliased in the input; one with any other slope will alternate between two slopes in the output. It will also smooth very tight curves. Unlike 2xSaI, it anti-aliases the output. hqnx was initially created for the Super NES emulator ZSNES. The author of bsnes has released a space-efficient implementation of hq2x to the public domain. A port to shaders, which has comparable quality to the early versions of xBR, is available. Before the port, a shader called "scalehq" has often been confused for hqx. === xBR family === There are 6 filters in this family: xBR , xBRZ, xBR-Hybrid, Super xBR, xBR+3D and Super xBR+3D. xBR ("scale by rules"), cre
Hubert Dreyfus's views on artificial intelligence
Hubert Dreyfus was a critic of artificial intelligence research. In a series of papers and books, including Alchemy and AI (1965), What Computers Can't Do (1972; 1979; 1992) and Mind over Machine (1986), he presented a skeptical and cautious assessment of AI's progress and a critique of the philosophical foundations of the field. Dreyfus' objections are discussed in most introductions to the philosophy of artificial intelligence, including Russell & Norvig (2021), a standard AI textbook, and in Fearn (2007), a survey of contemporary philosophy. Dreyfus argued that human intelligence and expertise depend primarily on yet-to-be understood informal and unconscious processes rather than symbolic manipulation and that these essentially human skills cannot be fully captured in formal rules. His critique was based on the insights of modern continental philosophers such as Merleau-Ponty and Heidegger, and was directed at the first wave of AI research which tried to reduce intelligence to high level formal symbols. When Dreyfus' ideas were first introduced in the mid-1960s, they were met in the AI community with ridicule and outright hostility. By the 1980s, however, some of his perspectives were rediscovered by researchers working in robotics and the new field of connectionism—approaches that were called "sub-symbolic" at the time because they eschewed early AI research's emphasis on high level symbols. In the 21st century, "sub-symbolic" artificial neural networks and other statistics-based approaches to machine learning were highly successful. Historian and AI researcher Daniel Crevier wrote: "time has proven the accuracy and perceptiveness of some of Dreyfus's comments." Dreyfus said in 2007, "I figure I won and it's over—they've given up." == Dreyfus' critique == === The grandiose promises of artificial intelligence === In Alchemy and AI (1965) and What Computers Can't Do (1972), Dreyfus summarized the history of artificial intelligence and ridiculed the unbridled optimism that permeated the field. For example, Herbert A. Simon, following the success of his program General Problem Solver (1957), predicted that by 1967: A computer would be world champion in chess. A computer would discover and prove an important new mathematical theorem. Most theories in psychology will take the form of computer programs. The press dutifully reported these predictions of the imminent arrival of machine intelligence. Dreyfus felt that this optimism was unwarranted and, in 1965, argued forcefully that predictions like these would not come true. He would eventually be proven right. Pamela McCorduck explains Dreyfus' position: A great misunderstanding accounts for public confusion about thinking machines, a misunderstanding perpetrated by the unrealistic claims researchers in AI have been making, claims that thinking machines are already here, or at any rate, just around the corner. These predictions were based on the success of the cognitive revolution, which promoted an "information processing" model of the mind. It was articulated by Newell and Simon in their physical symbol systems hypothesis, and later expanded into a philosophical position known as computationalism by philosophers such as Jerry Fodor and Hilary Putnam. In AI, the approach is now called symbolic AI or "GOFAI". Dreyfus argued that "symbolic AI" was the latest version of the ancient program of rationalism in philosophy. Rationalism had come under heavy criticism in the 20th century from philosophers like Martin Heidegger and Edmund Husserl. The mind, according to modern continental philosophy, is not "rationalist" and is nothing like a digital computer. Cognitivism led early AI researchers to believe that they had successfully simulated the essential process of human thought, thus it seemed a short step to producing fully intelligent machines. Dreyfus' last paper detailed the ongoing history of the "first step fallacy", where AI researchers tend to wildly extrapolate initial success as promising, perhaps even guaranteeing, wild future successes. === Dreyfus' four assumptions of artificial intelligence research === In Alchemy and AI and What Computers Can't Do, Dreyfus identified four philosophical assumptions, at least one of which he deems necessary for AI to succeed. "In each case," Dreyfus writes, "the assumption is taken by workers in AI as an axiom, guaranteeing results, whereas it is, in fact, one hypothesis among others, to be tested by the success of such work." Dreyfus argues that AI would be impossible without accepting at least one of these four assumptions: The biological assumption The brain processes information in discrete operations by way of some biological equivalent of on/off switches. In the early days of research into neurology, scientists found that neurons fire in all-or-nothing pulses. Several researchers, such as Walter Pitts and Warren McCulloch, speculated with great confidence that neurons functioned similarly to the way Boolean logic gates operate, and so could be imitated by electronic circuitry at the level of the neuron. When digital computers became widely used in the early 50s, this argument was extended to suggest that the brain was a vast physical symbol system, manipulating the binary symbols of zero and one. Dreyfus was able to refute the biological assumption by citing research in neurology that suggested that the action and timing of neuron firing had analog components. But Daniel Crevier observes that "few still held that belief in the early 1970s, and nobody argued against Dreyfus" about the biological assumption. The psychological assumption The mind can be viewed as a device operating on bits of information according to formal rules. He refuted this assumption by showing that much of what we know about the world consists of complex attitudes or tendencies that make us lean towards one interpretation over another. He argued that, even when we use explicit symbols, we are using them against an unconscious and informal background including commonsense knowledge and that without this background our symbols cease to mean anything. This background, in Dreyfus' view, was not implemented in individual brains as explicit individual symbols with explicit individual meanings. The epistemological assumption All knowledge can be formalized. This concerns the philosophical issue of epistemology, or the study of knowledge. Even if we agree that the psychological assumption is false, AI researchers could still argue (as AI founder John McCarthy has) that it is possible for a symbol processing machine to represent all knowledge, regardless of whether human beings represent knowledge the same way. Dreyfus argued that there is no justification for this assumption, since so much of human knowledge is not symbolic or even expressible using formal constructs. The ontological assumption The world consists of independent facts that can be represented by independent symbols AI researchers (and futurists and science fiction writers) often assume that there is no limit to formal, scientific knowledge, because they assume that any phenomenon in the universe can be described by symbols or scientific theories. This assumes that everything that exists can be understood as objects, properties of objects, classes of objects, relations of objects, and so on: precisely those things that can be described by logic, language and mathematics. The study of being or existence is called ontology, and so Dreyfus calls this the ontological assumption. If this is false, then it raises doubts about what we can ultimately know and what intelligent machines will ultimately be able to help us to do. === Knowing-how vs. knowing-that: the primacy of intuition === In Mind Over Machine (1986), written (with his brother) during the heyday of expert systems, Dreyfus analyzed the difference between human expertise and the programs that claimed to capture it. This expanded on ideas from What Computers Can't Do, where he had made a similar argument criticizing the "cognitive simulation" school of AI research practiced by Allen Newell and Herbert A. Simon in the 1960s. Dreyfus argued that human problem solving and expertise depend on our background sense of the context, of what is important and interesting given the situation, rather than on the process of searching through combinations of possibilities to find what we need. Dreyfus would describe it in 1986 as the difference between "knowing-that" and "knowing-how", based on Heidegger's distinction of present-at-hand and ready-to-hand. Knowing-that is our conscious, step-by-step problem solving abilities. We use these skills when we encounter a difficult problem that requires us to stop, step back and search through ideas one at time. At moments like this, the ideas become very precise and simple: they become context free symbols, which we manipulate using logic and language. These are the skills that Newell and Simon had demonstrated with both psy
Visual hierarchy
Visual hierarchy, in Gestalt psychology, describes how particular elements in a visual field stand out more than others in a pattern, creating a perceived order of importance. Although it can occur naturally, the term is most often used in design—especially graphic design and cartography—where elements are arranged to appear more important than others. This order is created by the visual contrast between forms in a field of perception. Objects with highest contrast to their surroundings are recognized first by the human mind. == Evidence == There is some scientific evidence for visual hierarchy using eye tracking. For example, one study found that when people agree that a graphic design is good, they exhibit more similar eye movements; measured by the Fréchet distance. == Theory == The concept of visual hierarchy is based in Gestalt psychological theory, an early 20th-century German theory that proposes that the human brain has innate organizing tendencies that “structure individual elements, shapes or forms into a coherent, organized whole,” especially when processing visual information. The German word Gestalt translates into “form,” “pattern,” or “shape” in English. When an element in a visual field disconnects from the ‘whole’ created by the brain's perceptual organization, it “stands out” to the viewer. The shapes that disconnect most severely from their surroundings stand out the most. This is commonly encapsulated as the Von Restorff effect, which states that isolation attracts attention. === Physical characteristics === The brain distinguishes objects based on differences in their physical appearances. These characteristics fall into four categories: color, size, alignment, and character. Each type of contrast can be used to construct a visual hierarchy. The same characteristics are also sometimes categorized (especially among cartographers) according to the visual variables of Jacques Bertin. Color encompasses the hue, saturation, value, and perceived texture of forms. Dark figures will stand out on a light background, light figures will stand out on a dark background, brightly colored figures will stand out on a muted background, and so on. The fluorescent colors used for tennis balls and other sports equipment is intended to make them instantly stand out against almost any natural visual field. Size has a strong influence on visual hierarchy. Large elements typically attract attention, provided that they can be recognized as figures. Alignment is the arrangement of forms relative to one another. For example, items in the upper left corner of a page are often seen first (at least for those readers accustomed to western languages), the center of the field has prominence. Negative space can also be employed: a figure isolated among large amounts of white space will stand out more than one amid other figures. Character includes several kinds of contrasts based on shape. For example, complex patterns attract more attention than simple or predictable patterns, intricate shapes attract more attention than generalized ones. Even large-scale patterns can attract attention if they contrast with the pattern in the remainder of the visual field. Camouflage is an example of eliminating contrast in character in color and/or character specifically to reduce visual hierarchy. The "squint test" is often suggested as a simple, if unscientific, method to evaluate the visual hierarchy of a graphical product like a map or web page. When viewed out of focus (or from a great distance), the viewer is not distracted by details, but can only see overall (gestalt) patterns such as visual hierarchy. All of the above patterns, except some aspects of character, are recognizable by this method. == Application == Visual hierarchy is an important concept in the field of graphic design, a field that specializes in visual organization. Designers attempt to control visual hierarchy to guide the eye to information in a specific order for a specific purpose. One could compare visual hierarchy in graphic design to grammatical structure in writing in terms of the importance of each principle to these fields. === Cartography === In cartographic design, visual hierarchy is used to emphasize certain important features on a map over less important features. Typically, a map has a purpose that dictates a conceptual hierarchy of what should be more or less important, so one of the goals of the choice of map symbols is to match the visual hierarchy to the conceptual hierarchy. The Visual hierarchy of a map may apply to individual geographic features (such as making a single country stand out), to map layers of related features (e.g., making lakes stand out more than roads), and to the entire layout of map and non-map elements (e.g., making the title look more important than the scale bar). Like the main map elements, such features have weight, and the properties that apply to visual hierarchy of map layers also apply to other elements on the page. Size and alignment are the two main determinants of the visual hierarchy for these features. Cartographers often utilize principles of negative space and figure-ground contrast to design an appropriate visual hierarchy by employing contrast between unused space and layout features. === User experience design and behavioral design === In user experience design and behavioural design, such as web design, visual hierarchy is used to prioritize navigational structures and content, so that audiences focus on elements that facilitate system usage, or increases the chance that they notice content that contains psychological nudges. Color is one of many factors used in the design of a visual hierarchy, and a key factor due to the high salience of color perception.
Imageability
Imageability is a measure of how easily a physical object, word or environment will evoke a clear mental image in the mind of any person observing it. It is used in architecture and city planning, in psycholinguistics, and in automated computer vision research. In automated image recognition, training models to connect images with concepts that have low imageability can lead to biased and harmful results. == History and components == Kevin A. Lynch first introduced the term, "imageability" in his 1960 book, The Image of the City. In the book, Lynch argues cities contain a key set of physical elements that people use to understand the environment, orient themselves inside of it, and assign it meaning. Lynch argues the five key elements that impact the imageability of a city are Paths, Edges, Districts, Nodes, and Landmarks. Paths: channels in which people travel. Examples: streets, sidewalks, trails, canals, railroads. Edges: objects that form boundaries around space. Examples: walls, buildings, shoreline, curbstone, streets, and overpasses. Districts: medium to large areas people can enter into and out of that have a common set of identifiable characteristics. Nodes: large areas people can enter, that serve as the foci of the city, neighborhood, district, etc. Landmarks: memorable points of reference people cannot enter into. Examples: signs, mountains and public art. In 1914, half a century before The Image of the City was published, Paul Stern discussed a concept similar to imageability in the context of art. Stern, in Susan Langer's Reflections on Art, names the attribute that describes how vividly and intensely an artistic object could be experienced apparency. == In computer vision == Automated image recognition was developed by using machine learning to find patterns in large, annotated datasets of photographs, like ImageNet. Images in ImageNet are labelled using concepts in WordNet. Concepts that are easily expressed verbally, like "early", are seen as less "imageable" than nouns referring to physical objects like "leaf". Training AI models to associate concepts with low imageability with specific images can lead to problematic bias in image recognition algorithms. This has particularly been critiqued as it relates to the "person" category of WordNet and therefore also ImageNet. Trevor Pagan and Kate Crawford demonstrated in their essay "Excavating AI" and their art project ImageNet Roulette how this leads to photos of ordinary people being labelled by AI systems as "terrorists" or "sex offenders". Images in datasets are often labelled as having a certain level of imageability. As described by Kaiyu Yang, Fei-Fei Li and co-authors, this is often done following criteria from Allan Paivio and collaborators' 1968 psycholinguistic study of nouns. Yang el.al. write that dataset annotators tasked with labelling imageability "see a list of words and rate each word on a 1-7 scale from 'low imagery' to 'high imagery'. To avoid biased or harmful image recognition and image generation, Yang et.al. recommend not training vision recognition models on concepts with low imageability, especially when the concepts are offensive (such as sexual or racial slurs) or sensitive (their examples for this category include "orphan", "separatist", "Anglo-Saxon" and "crossover voter"). Even "safe" concepts with low imageability, like "great-niece" or "vegetarian" can lead to misleading results and should be avoided.
Thunderspy
Thunderspy is a type of security vulnerability, based on the Intel Thunderbolt 3 port, first reported publicly on 10 May 2020, that can result in an evil maid (i.e., attacker of an unattended device) attack gaining full access to a computer's information in about five minutes, and may affect millions of Apple, Linux and Windows computers, as well as any computers manufactured before 2019, and some after that. According to Björn Ruytenberg, the discoverer of the vulnerability, "All the evil maid needs to do is unscrew the backplate, attach a device momentarily, reprogram the firmware, reattach the backplate, and the evil maid gets full access to the laptop. All of this can be done in under five minutes." The malicious firmware is used to clone device identities which makes classical DMA attack possible. == History == The Thunderspy security vulnerabilities were first publicly reported by Björn Ruytenberg of Eindhoven University of Technology in the Netherlands on 10 May 2020. Thunderspy is similar to Thunderclap, another security vulnerability, reported in 2019, that also involves access to computer files through the Thunderbolt port. == Impact == The security vulnerability affects millions of Apple, Linux and Windows computers, as well as all computers manufactured before 2019, and some after that. However, this impact is restricted mainly to how precise a bad actor would have to be to execute the attack. Physical access to a machine with a vulnerable Thunderbolt controller is necessary, as well as a writable ROM chip for the Thunderbolt controller's firmware. Additionally, part of Thunderspy, specifically the portion involving re-writing the firmware of the controller, requires the device to be in sleep, or at least in some sort of powered-on state, to be effective. Machines that force power-off when the case is open may assist in resisting this attack to the extent that the feature (switch) itself resists tampering. Due to the nature of attacks that require extended physical access to hardware, it's unlikely the attack will affect users outside of a business or government environment. == Mitigation == The researchers claim there is no easy software solution, and may only be mitigated by disabling the Thunderbolt port altogether. However, the impacts of this attack (reading kernel level memory without the machine needing to be powered off) are largely mitigated by anti-intrusion features provided by many business machines. Intel claims enabling such features would substantially restrict the effectiveness of the attack. Microsoft's official security recommendations recommend disabling sleep mode while using BitLocker. Using hibernation in place of sleep mode turns the device off, mitigating potential risks of attack on encrypted data.
Pattern language
A pattern language is an organized and coherent set of patterns, each of which describes a problem and the core of a solution that can be used in many ways within a specific field of expertise. The term was coined by architect Christopher Alexander and popularized by his 1977 book A Pattern Language. A pattern language can also be an attempt to express the deeper wisdom of what brings aliveness within a particular field of human endeavor, through a set of interconnected patterns. Aliveness is one placeholder term for "the quality that has no name": a sense of wholeness, spirit, or grace, that while of varying form, is precise and empirically verifiable. Alexander claims that ordinary people can use this design approach to successfully solve very large, complex design problems. == What is a pattern? == When a designer designs something – whether a house, computer program, or lamp – they must make many decisions about how to solve problems. A single problem is documented with its typical place (the syntax), and use (the grammar) with the most common and recognized good solution seen in the wild, like the examples seen in dictionaries. Each such entry is a single design pattern. Each pattern has a name, a descriptive entry, and some cross-references, much like a dictionary entry. A documented pattern should explain why that solution is good in the pattern's contexts. Elemental or universal patterns such as "door" or "partnership" are versatile ideals of design, either as found in experience or for use as components in practice, explicitly described as holistic resolutions of the forces in recurrent contexts and circumstances, whether in architecture, medicine, software development or governance, etc. Patterns might be invented or found and studied, such as the naturally occurring patterns of design that characterize human environments. Like all languages, a pattern language has vocabulary, syntax, and grammar – but a pattern language applies to some complex activity other than communication. In pattern languages for design, the parts break down in this way: The language description – the vocabulary – is a collection of named, described solutions to problems in a field of interest. These are called design patterns. So, for example, the language for architecture describes items like: settlements, buildings, rooms, windows, latches, etc. Each solution includes syntax, a description that shows where the solution fits in a larger, more comprehensive or more abstract design. This automatically links the solution into a web of other needed solutions. For example, rooms have ways to get light, and ways to get people in and out. The solution includes grammar that describes how the solution solves a problem or produces a benefit. So, if the benefit is unneeded, the solution is not used. Perhaps that part of the design can be left empty to save money or other resources; if people do not need to wait to enter a room, a simple doorway can replace a waiting room. In the language description, grammar and syntax cross index (often with a literal alphabetic index of pattern names) to other named solutions, so the designer can quickly think from one solution to related, needed solutions, and document them in a logical way. In Christopher Alexander's book A Pattern Language, the patterns are in decreasing order by size, with a separate alphabetic index. The web of relationships in the index of the language provides many paths through the design process. This simplifies the design work because designers can start the process from any part of the problem they understand and work toward the unknown parts. At the same time, if the pattern language has worked well for many projects, there is reason to believe that even a designer who does not completely understand the design problem at first will complete the design process, and the result will be usable. For example, skiers coming inside must shed snow and store equipment. The messy snow and boot cleaners should stay outside. The equipment needs care, so the racks should be inside. == Many patterns form a language == Just as words must have grammatical and semantic relationships to each other in order to make a spoken language useful, design patterns must be related to each other in position and utility order to form a pattern language. Christopher Alexander's work describes a process of decomposition, in which the designer has a problem (perhaps a commercial assignment), selects a solution, then discovers new, smaller problems resulting from the larger solution. Occasionally, the smaller problems have no solution, and a different larger solution must be selected. Eventually all of the remaining design problems are small enough or routine enough to be solved by improvisation by the builders, and the "design" is done. The actual organizational structure (hierarchical, iterative, etc.) is left to the discretion of the designer, depending on the problem. This explicitly lets a designer explore a design, starting from some small part. When this happens, it's common for a designer to realize that the problem is actually part of a larger solution. At this point, the design almost always becomes a better design. In the language, therefore, each pattern has to indicate its relationships to other patterns and to the language as a whole. This gives the designer using the language a great deal of guidance about the related problems that must be solved. The most difficult part of having an outside expert apply a pattern language is in fact to get a reliable, complete list of the problems to be solved. Of course, the people most familiar with the problems are the people that need a design. So, Alexander famously advocated on-site improvisation by concerned, empowered users, as a powerful way to form very workable large-scale initial solutions, maximizing the utility of a design, and minimizing the design rework. The desire to empower users of architecture was, in fact, what led Alexander to undertake a pattern language project for architecture in the first place. == Design problems in a context == An important aspect of design patterns is to identify and document the key ideas that make a good system different from a poor system (that may be a house, a computer program or an object of daily use), and to assist in the design of future systems. The idea expressed in a pattern should be general enough to be applied in very different systems within its context, but still specific enough to give constructive guidance. The range of situations in which the problems and solutions addressed in a pattern apply is called its context. An important part in each pattern is to describe this context. Examples can further illustrate how the pattern applies to very different situation. For instance, Alexander's pattern "A PLACE TO WAIT" addresses bus stops in the same way as waiting rooms in a surgery, while still proposing helpful and constructive solutions. The "Gang-of-Four" book Design Patterns by Gamma et al. proposes solutions that are independent of the programming language, and the program's application domain. Still, the problems and solutions described in a pattern can vary in their level of abstraction and generality on the one side, and specificity on the other side. In the end this depends on the author's preferences. However, even a very abstract pattern will usually contain examples that are, by nature, absolutely concrete and specific. Patterns can also vary in how far they are proven in the real world. Alexander gives each pattern a rating by zero, one or two stars, indicating how well they are proven in real-world examples. It is generally claimed that all patterns need at least some existing real-world examples. It is, however, conceivable to document yet unimplemented ideas in a pattern-like format. The patterns in Alexander's book also vary in their level of scale – some describing how to build a town or neighbourhood, others dealing with individual buildings and the interior of rooms. Alexander sees the low-scale artifacts as constructive elements of the large-scale world, so they can be connected to a hierarchic network. === Balancing of forces === A pattern must characterize the problems that it is meant to solve, the context or situation where these problems arise, and the conditions under which the proposed solutions can be recommended. Often these problems arise from a conflict of different interests or "forces". A pattern emerges as a dialogue that will then help to balance the forces and finally make a decision. For instance, there could be a pattern suggesting a wireless telephone. The forces would be the need to communicate, and the need to get other things done at the same time (cooking, inspecting the bookshelf). A very specific pattern would be just "WIRELESS TELEPHONE". More general patterns would be "WIRELESS DEVICE" or "SECONDARY ACTIVITY", suggesting that a secondary activity (such as talking on t