Subpixel rendering

Subpixel rendering

Subpixel rendering is a method used to increase the effective resolution of a color display device. It utilizes the composition of each pixel, which consists of three subpixels of which are red, green, and blue that can each be individually addressable on the display matrix. Subpixel rendering is primarily used for text rendering on standard DPI displays. Despite the inherent color anomalies, it can also be used to render general graphics. == History == The origin of subpixel rendering as used today remains controversial. Apple Inc., IBM, and Microsoft patented various implementations that differed in technical details owing to the different purposes for which their technologies were intended. Microsoft held several patents in the United States for subpixel rendering technology used in text rendering on RGB Stripe layouts. The patents 6,219,025; 6,239,783; 6,307,566; 6,225,973; 6,243,070; 6,393,145; 6,421,054; 6,282,327; and 6,624,828 were filed between October 7, 1998, and October 7, 1999, and expired on July 30, 2019. Analysis of the patent by FreeType indicates that the patent does not cover the idea of subpixel rendering, but rather the actual filter used as a last step to balance the color. Microsoft's patent describes the smallest possible filter that distributes each subpixel value equally among the R, G, and B pixels. Any other filter will either be blurrier or will introduce color artifacts. Apple was able to use it in Mac OS X due to a patent cross-licensing agreement. == Characteristics == A single pixel on a color display is made of several subpixels, typically three arranged left-to-right as red, green, and blue (RGB). The components are readily visible with a small magnifying glass, such as a loupe. These pixel components appear as a single color to the human eye because of blurring by optics and spatial integration by nerve cells in the eye. However, the eye is much more sensitive to the location. Therefore, turning on the G and B of one pixel and the R of the next pixel to the right will produce a white dot, but it will appear to be 1/3 of a pixel to the right of the white dot that would be seen from the RGB of only the first pixel. Subpixel rendering leverages this to provide three times the horizontal resolution of the rendered image. However, it has to blur this image to produce the correct color by ensuring the same amount of red, green, and blue are turned on as when no subpixel rendering is being done. Subpixel rendering does not necessitate the use of antialiasing. It gives a smoother result regardless of whether antialiasing is used or not since it artificially increases the resolution. However, it introduces color aliasing since subpixels are colored. Subsequent filtering applied to remove the color artifacts is a form of antialiasing, although its purpose is not smoothing jagged shapes as in conventional antialiasing. Subpixel rendering requires the software to know the layout of the subpixels. The most common reason it is wrong is monitors that can be rotated 90 (or 180) degrees, though monitors are manufactured with other arrangements of the subpixels, such as BGR or in triangles, or with 4 colors like RGBW squares. On any such display the result of incorrect subpixel rendering will be worse than if no subpixel rendering was done at all (it will not produce color artifacts, but it will produce noisy edges). == Implementations == === Apple II === Steve Gibson has claimed that the Apple II, introduced in 1977, supports an early form of subpixel rendering in its high-resolution (280×192) graphics mode. The Wozniak patent only used 2 "sub-pixels". The bytes that comprise the Apple II high-resolution screen buffer contain seven visible bits (each corresponding directly to a pixel) and a flag bit used to select between purple/green or blue/orange color sets. Each pixel, since it is represented by a single bit, is either on or off; there are no bits within the pixel itself for specifying color or brightness. Color is instead created as an artifact of the NTSC color encoding scheme, determined by horizontal position: pixels with even horizontal coordinates are always purple (or blue, if the flag bit is set), and odd pixels are always green (or orange). Two lit pixels next to each other are always white, regardless of whether the pair is even/odd or odd/even, and irrespective of the value of the flag bit. This is an approximation, but it is what most programmers of the time would have in mind while working with the Apple's high-resolution mode. Gibson's example claims that because two adjacent bits form a white block, there are, in fact, two bits per pixel: one that activates the pixel's purple left half and the other that activates its green right half. If the programmer instead activates the green right half of a pixel and the purple left half of the next pixel, the result is a white block 1/2 pixel to the right, which is indeed an instance of subpixel rendering. However, it is not clear whether any programmers of the Apple II have considered the pairs of bits as pixels—instead calling each bit a pixel. The flag bit in each byte affects color by shifting pixels half a pixel-width to the right. This half-pixel shift was exploited by some graphics software, such as HRCG (High-Resolution Character Generator), an Apple utility that displayed text using the high-resolution graphics mode, to smooth diagonals. === ClearType === Microsoft announced its subpixel rendering technology, called ClearType, at COMDEX in 1998. Microsoft published a paper in May 2000, Displaced Filtering for Patterned Displays, describing the filtering behind ClearType. It was then made available in Windows XP. Still, it was not activated by default until Windows Vista, while Windows XP OEMs could and did change the default setting. === FreeType === FreeType, the library used by most current software on the X Window System, contains two open source implementations. The original implementation uses the ClearType antialiasing filters and carries the following notice: "The colour filtering algorithm of Microsoft's ClearType technology for subpixel rendering is covered by patents; for this reason, the corresponding code in FreeType is disabled by default. Note that subpixel rendering per se is prior art; using a different colour filter thus easily circumvents Microsoft's patent claims." FreeType offers a variety of color filters. Since version 2.6.2, the default filter is light, a filter that is both normalized (value sums up to 1) and color-balanced (eliminate color fringes at the cost of resolution). Since version 2.8.1, a second implementation exists, called Harmony, that "offers high quality LCD-optimized output without resorting to ClearType techniques of resolution tripling and filtering". This is the method enabled by default. When using this method, "each color channel is generated separately after shifting the glyph outline, capitalizing on the fact that the color grids on LCD panels are shifted by a third of a pixel. This output is indistinguishable from ClearType with a light 3-tap filter." Since the Harmony method does not require additional filtering, it is not covered by the ClearType patents. === CoolType === Adobe created their own subpixel renderer called CoolType, allowing them to display documents the same way across various operating systems: Windows, MacOS, Linux etc. When it was launched around the year 2001, CoolType supported a wider range of fonts than Microsoft's ClearType, which at the time was limited to TrueType fonts. In contrast, Adobe's CoolType also supported PostScript fonts (and their OpenType equivalents). === macOS === Mac OS X (later OS X, now macOS) also used subpixel rendering, as part of Quartz 2D. However, it was removed after the introduction of Retina displays. Unlike Microsoft's implementation, which favors a tight fit to the grid (font hinting) to maximize legibility, Apple's implementation prioritizes the shape of the glyphs as set out by their designer.

Computer appliance

A computer appliance is a computer system with a combination of hardware, software, or firmware that is specifically designed to provide a particular computing resource. Such devices became known as appliances because of the similarity in role or management to a home appliance, which are generally closed and sealed, and are not serviceable by the user or owner. The hardware and software are delivered as an integrated product and may even be pre-configured before delivery to a customer, to provide a turn-key solution for a particular application. Unlike general purpose computers, appliances are generally not designed to allow the customers to change the software and the underlying operating system, or to flexibly reconfigure the hardware. Another form of appliance is the virtual appliance, which has similar functionality to a dedicated hardware appliance, but is distributed as a software virtual machine image for a hypervisor-equipped device. == Overview == Traditionally, software applications run on top of a general-purpose operating system, which uses the hardware resources of the computer (primarily memory, disk storage, processing power, and networking bandwidth) to meet the computing needs of the user. The main issue with the traditional model is related to complexity. It is complex to integrate the operating system and applications with a hardware platform, and complex to support it afterwards. By tightly constraining the variations of the hardware and software, the appliance becomes easily deployable, and can be used without nearly as wide (or deep) IT knowledge. Additionally, when problems and errors appear, the supporting staff very rarely needs to explore them deeply to understand the matter thoroughly. The staff needs merely training on the appliance management software to be able to resolve most of problems. In all forms of the computer appliance model, customers benefit from easy operations. The appliance has exactly one combination of hardware and operating system and application software, which has been pre-installed at the factory. This prevents customers from needing to perform complex integration work, and dramatically simplifies troubleshooting. In fact, this "turnkey operation" characteristic is the driving benefit that customers seek when purchasing appliances. To be considered an appliance, the (hardware) device needs to be integrated with software, and both are supplied as a package. This distinguishes appliances from "home grown" solutions, or solutions requiring complex implementations by integrators or value-added resellers (VARs). The appliance approach helps to decouple the various systems and applications, for example in the data center. Once a resource is decoupled, in theory it can be also centralized to become shared among many systems, centrally managed and optimized, all without requiring changes to any other system. == Tradeoffs of the computer appliance approach == The major disadvantage of deploying a computer appliance is that since they are designed to supply a specific resource, they most often include a customized operating system running over specialized hardware, neither of which are likely to be compatible with the other systems previously deployed. Customers lose flexibility. One may believe that a proprietary embedded operating system, or operating system within an application, can make the appliance much more secure from common cyber attacks. However, the opposite is true. Security by obscurity is a poor security decision, and appliances are often plagued by security issues as evidenced by the proliferation of IoT devices. == Types of appliances == The variety of computer appliances reflects the wide range of computing resources they provide to applications. Some examples: Storage appliances provide large amounts of storage, often available to many machines on the network. See Network-attached storage and Storage area network. Network appliances are general purpose routers which may also provide firewall protection, Transport Layer Security (TLS), messaging, access to specialized networking protocols (like the ebXML Message Service) and bandwidth multiplexing for the multiple systems they front-end. Backup and disaster recovery appliances computer appliances that are integrated backup software and backup targets, sometimes with hypervisors to support local DR of protected servers. They are often a gateway to a full DRaaS solution. Firewall and Security appliances Dedicated network appliances that are designed to protect computer networks from unwanted traffic. IIoT and MES Gateway appliances Computer appliances that are designed to translate data bidirectionally between control systems and enterprise systems. Proprietary, embedded, firmware applications running on the appliance use point-to-point connections to translate data between field devices in their native automation protocols and MES systems through their APIs, ODBC, or RESTful interfaces. Anti-spam appliances for e-mail spam Software appliances A single application server appliance, with just enough operating system (JeOS) for it to run. Virtual machine appliances consist of a "hypervisor style" embedded operating system running on appliance hardware. The hypervisor layer is matched to the hardware of the appliance, and cannot be varied by the customer, but the customer may load other operating systems and applications onto the appliance in the form of virtual machines. == Consumer appliances == Aside from its deployment within data centers, many computer appliances are directly used by the general public. These include: Digital video recorder Residential gateway Network-attached storage (NAS) Video game console Consumer uses stress the need for an appliance to have easy installation, configuration, and operation, with little or no technical knowledge being necessary. == Appliances in industrial automation == The world of industrial automation has been rich in appliances. These appliances have been hardened to withstand temperature and vibration extremes. These appliances are also highly configurable, enabling customization to meet a wide variety of applications. The key benefits of an appliance in automation are: Reduced downtime - a failed appliance is typically replaced with a COTS replacement and its task is quickly and easily reloaded from a backup. Highly scalable - appliances are typically targeted solutions for an area of a plant or process. As the requirements change, scalability is achieved through the installation of another appliance. Automation concepts are easily replicated throughout the enterprise by standardizing on appliances to perform the needed tasks, as opposed to the development of custom automation schemes for each task. Low TCO (total cost of ownership) - appliances are developed, tested and supported by automation product vendors and undergo a much broader level of quality testing than custom designed automation solutions. The use of appliances in automation reduce the level of testing needed in each individual application. Reduced design time - appliances perform specific functions and although they are highly configurable, they are typically self documenting. This enables appliance based solutions to be transferred from engineer to engineer with minimal need for training and documentation. Types of automation appliances: PLC (programmable logic controller) - Programmable logic controllers are appliances that are typically used for discrete control and offer a wide range of Input and Output options. They are configured through standardized programming languages such as IEC-1131. PID (proportional–integral–derivative controller) - PID controllers are appliances that monitor a process variable and, based on an error term, effect change on a control output (manipulated variable) to drive the process variable to a setpoint. PAC (programmable automation controller) - Programmable automation controllers are appliances that embody properties of both PLCs and PID controllers enabling the integration of both analog and discrete control. Universal gateway - A universal gateway appliance has the ability to communicate with a variety of devices through their respective communication protocols, and will affect data transactions between them. This in increasingly important as manufacturing strives to improve agility, quality, production rates, production costs and reduce downtime through enhanced M2M (machine to machine) communications. EATMs (Enterprise Appliance Transaction Modules) - Enterprise appliance transaction modules are appliances that affect data transactions from plant floor automation systems to enterprise business systems. They communicate to plant floor equipment through various vendor automation protocols, and communicate to business systems through database communication protocols such as JMS (Java Message Service) and SQL (Structured Query Language). == Internal structure == There are several

Nona-binning

Nona-binning is a pixel binning technique used in high-resolution image sensors, primarily in smartphone cameras. The method is based on merging groups of nine neighbouring pixels arranged in a 3×3 pattern. This configuration allows a sensor with very small individual pixels to increase its effective light sensitivity when operating in low-light conditions, while still maintaining high nominal resolution in bright environments. == Overview == Nona-binning is most commonly implemented in sensors with a resolution of 108 megapixels and higher. As pixel counts grew, the physical dimensions of individual pixels continued to shrink, reducing the amount of light captured by each. The 3×3 binning structure enables a sensor to operate in two modes. In well-lit scenes, each pixel is processed separately, providing the full resolution of the sensor. In darker settings, nine pixels with identical colour filters are combined into a single output unit, increasing signal strength and reducing noise. == Technical principles == Unlike the traditional Bayer colour filter array, which alternates colours on a per-pixel basis, nona-binning uses a grouped layout. The sensor forms blocks of nine pixels with matching colour filters — typically within a Quad Bayer–derived arrangement extended to 3×3 regions. When operating in the binning mode, the sensor aggregates the charge generated by all nine pixels in each block. This increases effective sensitivity but lowers the final image resolution. When lighting conditions allow, the sensor returns to processing pixel data individually. == Applications == Nona-binning is primarily used in: Smartphone photography, particularly in devices equipped with sensors exceeding 100 megapixels. Low-light imaging, where increased sensitivity improves exposure stability and reduces noise. Computational photography systems, such as multi-frame processing and HDR capture. == Related technologies == Nona-binning belongs to the broader group of pixel-binning approaches used in modern sensors. Other implementations include Tetracell, which merges four pixels in a 2×2 block, and hexa-binning, which combines six pixels, though it is less common. All of these methods aim to balance the high nominal resolution of mobile sensors with the need for improved low-light performance.

Tiimo

Tiimo is an app designed to help neurodivergent individuals with planning their life. In August 2024 the company raised €1.4 million, bringing their total funding to €4.3 million. At that point they had over 500,000 users, including 50,000 paid users. The app has Apple Watch support and a learning platform that includes courses on well-being and neurodiversity. The app was founded by Helene Lassen Nørlem and Melissa Würtz Azari in 2015. After being a finalist in 2024, in December 2025 Tiimo was won Apple’s iPhone App of the Year. The premium version is $10/mo and features an AI chatbot alongside the daily planner.

Elowan

Elowan is a plant-robot cyborg. Using its own internal bioelectrical signals, The plant has a robotic extension that makes it move towards light sources. Electrodes are inserted into the leaves, stem, and ground to detect the faint bioelectrical signals the plant produces. Then they are amplified so the robot can read them. So when the plant "wants" to go to light, the cyborg automatically goes to the nearest light source. Future extensions of the robot could provide: Protection, growth frameworks, and nutrients. Other factors that could make the cyborg move are temperature, soil, and gravity conditions Elowan is one in a series of plant-electronic hybrid experiments.

Plotting algorithms for the Mandelbrot set

There are many programs and algorithms used to plot the Mandelbrot set and other fractals, some of which are described in fractal-generating software. These programs use a variety of algorithms to determine the color of individual pixels efficiently. == Escape time algorithm == The simplest algorithm for generating a representation of the Mandelbrot set is known as the "escape time" algorithm. A repeating calculation is performed for each x, y point in the plot area and based on the behavior of that calculation, a color is chosen for that pixel. === Unoptimized naïve escape time algorithm === In both the unoptimized and optimized escape time algorithms, the x and y locations of each point are used as starting values in a repeating, or iterating calculation (described in detail below). The result of each iteration is used as the starting values for the next. The values are checked during each iteration to see whether they have reached a critical "escape" condition, or "bailout". If that condition is reached, the calculation is stopped, the pixel is drawn, and the next x, y point is examined. For some starting values, escape occurs quickly, after only a small number of iterations. For starting values very close to but not in the set, it may take hundreds or thousands of iterations to escape. For values within the Mandelbrot set, escape will never occur. The programmer or user must choose how many iterations–or how much "depth"–they wish to examine. The higher the maximal number of iterations, the more detail and subtlety emerge in the final image, but the longer time it will take to calculate the fractal image. Escape conditions can be simple or complex. Because no complex number with a real or imaginary part greater than 2 can be part of the set, a common bailout is to escape when either coefficient exceeds 2. A more computationally complex method that detects escapes sooner, is to compute distance from the origin using the Pythagorean theorem, i.e., to determine the absolute value, or modulus, of the complex number. If this value exceeds 2, or equivalently, when the sum of the squares of the real and imaginary parts exceed 4, the point has reached escape. More computationally intensive rendering variations include the Buddhabrot method, which finds escaping points and plots their iterated coordinates. The color of each point represents how quickly the values reached the escape point. Often black is used to show values that fail to escape before the iteration limit, and gradually brighter colors are used for points that escape. This gives a visual representation of how many cycles were required before reaching the escape condition. To render such an image, the region of the complex plane we are considering is subdivided into a certain number of pixels. To color any such pixel, let c {\displaystyle c} be the midpoint of that pixel. We now iterate the critical point 0 under P c {\displaystyle P_{c}} , checking at each step whether the orbit point has modulus larger than 2. When this is the case, we know that c {\displaystyle c} does not belong to the Mandelbrot set, and we color our pixel according to the number of iterations used to find out. Otherwise, we keep iterating up to a fixed number of steps, after which we decide that our parameter is "probably" in the Mandelbrot set, or at least very close to it, and color the pixel black. In pseudocode, this algorithm would look as follows. The algorithm does not use complex numbers and manually simulates complex-number operations using two real numbers, for those who do not have a complex data type. The program may be simplified if the programming language includes complex-data-type operations. for each pixel (Px, Py) on the screen do x0 := scaled x coordinate of pixel (scaled to lie in the Mandelbrot X scale (-2.00, 0.47)) y0 := scaled y coordinate of pixel (scaled to lie in the Mandelbrot Y scale (-1.12, 1.12)) x := 0.0 y := 0.0 iteration := 0 max_iteration := 1000 while (xx + yy ≤ 22 AND iteration < max_iteration) do xtemp := xx - yy + x0 y := 2xy + y0 x := xtemp iteration := iteration + 1 color := palette[iteration] plot(Px, Py, color) Here, relating the pseudocode to c {\displaystyle c} , z {\displaystyle z} and P c {\displaystyle P_{c}} : z = x + i y {\displaystyle z=x+iy\ } z 2 = x 2 + 2 i x y {\displaystyle z^{2}=x^{2}+2ixy} - y 2 {\displaystyle y^{2}\ } c = x 0 + i y 0 {\displaystyle c=x_{0}+iy_{0}\ } and so, as can be seen in the pseudocode in the computation of x and y: x = R e ⁡ ( z 2 + c ) = x 2 − y 2 + x 0 {\displaystyle x=\mathop {\mathrm {Re} } (z^{2}+c)=x^{2}-y^{2}+x_{0}} and y = I m ⁡ ( z 2 + c ) = 2 x y + y 0 . {\displaystyle y=\mathop {\mathrm {Im} } (z^{2}+c)=2xy+y_{0}.\ } To get colorful images of the set, the assignment of a color to each value of the number of executed iterations can be made using one of a variety of functions (linear, exponential, etc.). One practical way, without slowing down calculations, is to use the number of executed iterations as an entry to a palette initialized at startup. If the color table has, for instance, 500 entries, then the color selection is n mod 500, where n is the number of iterations. === Optimized escape time algorithms === The code in the previous section uses an unoptimized inner while loop for clarity. In the unoptimized version, one must perform five multiplications per iteration. To reduce the number of multiplications the following code for the inner while loop may be used instead: x2:= 0 y2:= 0 w:= 0 while (x2 + y2 ≤ 4 and iteration < max_iteration) do x:= x2 - y2 + x0 y:= w - x2 - y2 + y0 x2:= x x y2:= y y w:= (x + y) (x + y) iteration:= iteration + 1 The above code works via some algebraic simplification of the complex multiplication: ( i y + x ) 2 = − y 2 + 2 i y x + x 2 = x 2 − y 2 + 2 i y x {\displaystyle {\begin{aligned}(iy+x)^{2}&=-y^{2}+2iyx+x^{2}\\&=x^{2}-y^{2}+2iyx\end{aligned}}} Using the above identity, the number of multiplications can be reduced to three instead of five. The above inner while loop can be further optimized by expanding w to w = x 2 + 2 x y + y 2 {\displaystyle w=x^{2}+2xy+y^{2}} Substituting w into y = w − x 2 − y 2 + y 0 {\displaystyle y=w-x^{2}-y^{2}+y_{0}} yields y = 2 x y + y 0 {\displaystyle y=2xy+y_{0}} and hence calculating w is no longer needed. The further optimized pseudocode for the above is: x:= 0 y:= 0 x2:= 0 y2:= 0 while (x2 + y2 ≤ 4 and iteration < max_iteration) do x2:= x x y2:= y y y:= 2 x y + y0 x:= x2 - y2 + x0 iteration:= iteration + 1 Note that in the above pseudocode, 2 x y {\displaystyle 2xy} seems to increase the number of multiplications by 1, but since 2 is the multiplier the code can be optimized via ( x + x ) y {\displaystyle (x+x)y} . == Coloring algorithms == In addition to plotting the set, a variety of algorithms have been developed to efficiently color the set in an aesthetically pleasing way show structures of the data (scientific visualisation) === Histogram coloring === A more complex coloring method involves using a histogram which pairs each pixel with said pixel's maximum iteration count before escape/bailout. This method will equally distribute colors to the same overall area, and, importantly, is independent of the maximum number of iterations chosen. This algorithm has four passes. The first pass involves calculating the iteration counts associated with each pixel (but without any pixels being plotted). These are stored in an array IterationCounts[x][y], where x and y are the x and y coordinates of said pixel on the screen respectively. The first step of the second pass is to create an array NumIterationsPerPixel[n], where the array size n is the maximum iteration count. Next, one must iterate over the array of pixel-iteration count pairs IterationCounts[x][y], and retrieve each pixel's saved iteration count, i, via e.g. i = IterationCounts[x][y]. After each pixel's iteration count i is retrieved, it is necessary to index the NumIterationsPerPixel array at i and increment the indexed value (which is initially zero) -- e.g. NumIterationsPerPixel[i] = NumIterationsPerPixel[i] + 1. for (x = 0; x < width; x++) do for (y = 0; y < height; y++) do i:= IterationCounts[x][y] NumIterationsPerPixel[i]++ The third pass iterates through the NumIterationsPerPixel array and adds up all the stored values, saving them in total. The array index represents the number of pixels that reached that iteration count before bailout. total: = 0 for (i = 0; i < max_iterations; i++) do total += NumIterationsPerPixel[i] After this, the fourth pass begins and all the values in the IterationCounts array are indexed, and, for each iteration count i, associated with each pixel, the count is added to a global sum of all the iteration counts from 1 to i in the NumIterationsPerPixel array . This value is then normalized by dividing the sum by the total value computed earlier. hue[][]:= 0.0 for (x = 0; x < width; x++) do for (y = 0; y < height; y++) do iteration:= Iteration

RIPAC (microprocessor)

RIPAC was a VLSI single-chip microprocessor designed for automatic recognition of the connected speech, one of the first of this use. The project of the microprocessor RIPAC started in 1984. RIPAC was aimed to provide efficient real-time speech recognition services to the italian telephone system provided by SIP. The microprocessor was presented in September 1986 at The Hague (Netherlands) at EUSPICO conference. It was composed of 70.000 transistors and structured as Harvard architecture. The name RIPAC is the acronym for "Riconoscimento del PArlato Connesso", that means "Recognition of the connected speech" in Italian. The microprocessor was designed by the Italian companies CSELT and ELSAG and was produced by SGS: a combination of Hidden Markov Model and Dynamic Time Warping algorithms was used for processing speech signals. It was able to do real-time speech recognition of Italian and many languages with a good affordability. The chip, issued by U.S. Patent No. 4,907,278, worked at first run.