Maximum-entropy Markov model

Maximum-entropy Markov model

In statistics, a maximum-entropy Markov model (MEMM), or conditional Markov model (CMM), is a graphical model for sequence labeling that combines features of hidden Markov models (HMMs) and maximum entropy (MaxEnt) models. An MEMM is a discriminative model that extends a standard maximum entropy classifier by assuming that the unknown values to be learnt are connected in a Markov chain rather than being conditionally independent of each other. MEMMs find applications in natural language processing, specifically in part-of-speech tagging and information extraction. == Model == Suppose we have a sequence of observations O 1 , … , O n {\displaystyle O_{1},\dots ,O_{n}} that we seek to tag with the labels S 1 , … , S n {\displaystyle S_{1},\dots ,S_{n}} that maximize the conditional probability P ( S 1 , … , S n ∣ O 1 , … , O n ) {\displaystyle P(S_{1},\dots ,S_{n}\mid O_{1},\dots ,O_{n})} . In a MEMM, this probability is factored into Markov transition probabilities, where the probability of transitioning to a particular label depends only on the observation at that position and the previous position's label: P ( S 1 , … , S n ∣ O 1 , … , O n ) = ∏ t = 1 n P ( S t ∣ S t − 1 , O t ) . {\displaystyle P(S_{1},\dots ,S_{n}\mid O_{1},\dots ,O_{n})=\prod _{t=1}^{n}P(S_{t}\mid S_{t-1},O_{t}).} Each of these transition probabilities comes from the same general distribution P ( s ∣ s ′ , o ) {\displaystyle P(s\mid s',o)} . For each possible label value of the previous label s ′ {\displaystyle s'} , the probability of a certain label s {\displaystyle s} is modeled in the same way as a maximum entropy classifier: P ( s ∣ s ′ , o ) = P s ′ ( s ∣ o ) = 1 Z ( o , s ′ ) exp ⁡ ( ∑ a λ a f a ( o , s ) ) . {\displaystyle P(s\mid s',o)=P_{s'}(s\mid o)={\frac {1}{Z(o,s')}}\exp \left(\sum _{a}\lambda _{a}f_{a}(o,s)\right).} Here, the f a ( o , s ) {\displaystyle f_{a}(o,s)} are real-valued or categorical feature-functions, and Z ( o , s ′ ) {\displaystyle Z(o,s')} is a normalization term ensuring that the distribution sums to one. This form for the distribution corresponds to the maximum entropy probability distribution satisfying the constraint that the empirical expectation for the feature is equal to the expectation given the model: E e ⁡ [ f a ( o , s ) ] = E p ⁡ [ f a ( o , s ) ] for all a . {\displaystyle \operatorname {E} _{e}\left[f_{a}(o,s)\right]=\operatorname {E} _{p}\left[f_{a}(o,s)\right]\quad {\text{ for all }}a.} The parameters λ a {\displaystyle \lambda _{a}} can be estimated using generalized iterative scaling. Furthermore, a variant of the Baum–Welch algorithm, which is used for training HMMs, can be used to estimate parameters when training data has incomplete or missing labels. The optimal state sequence S 1 , … , S n {\displaystyle S_{1},\dots ,S_{n}} can be found using a very similar Viterbi algorithm to the one used for HMMs. The dynamic program uses the forward probability: α t + 1 ( s ) = ∑ s ′ ∈ S α t ( s ′ ) P s ′ ( s ∣ o t + 1 ) . {\displaystyle \alpha _{t+1}(s)=\sum _{s'\in S}\alpha _{t}(s')P_{s'}(s\mid o_{t+1}).} == Strengths and weaknesses == An advantage of MEMMs rather than HMMs for sequence tagging is that they offer increased freedom in choosing features to represent observations. In sequence tagging situations, it is useful to use domain knowledge to design special-purpose features. In the original paper introducing MEMMs, the authors write that "when trying to extract previously unseen company names from a newswire article, the identity of a word alone is not very predictive; however, knowing that the word is capitalized, that is a noun, that it is used in an appositive, and that it appears near the top of the article would all be quite predictive (in conjunction with the context provided by the state-transition structure)." Useful sequence tagging features, such as these, are often non-independent. Maximum entropy models do not assume independence between features, but generative observation models used in HMMs do. Therefore, MEMMs allow the user to specify many correlated, but informative features. Another advantage of MEMMs versus HMMs and conditional random fields (CRFs) is that training can be considerably more efficient. In HMMs and CRFs, one needs to use some version of the forward–backward algorithm as an inner loop in training. However, in MEMMs, estimating the parameters of the maximum-entropy distributions used for the transition probabilities can be done for each transition distribution in isolation. A drawback of MEMMs is that they potentially suffer from the "label bias problem," where states with low-entropy transition distributions "effectively ignore their observations." Conditional random fields were designed to overcome this weakness, which had already been recognised in the context of neural network-based Markov models in the early 1990s. Another source of label bias is that training is always done with respect to known previous tags, so the model struggles at test time when there is uncertainty in the previous tag.

Artificial psychology

Artificial psychology (AP) has had multiple meanings dating back to 19th century, with recent usage related to artificial intelligence (AI).Artificial psychology is a theoretical field related to artificial intelligence, cognitive science, and psychology, which explores how advanced AI systems may develop human-like decision-making processes. In 1999, Zhiliang Wang and Lun Xie presented a theory of artificial psychology based on artificial intelligence. They analyze human psychology using information science research methods and artificial intelligence research to probe deeper into the human mind. == Main Theory == Dan Curtis (b. 1963) proposed AP is a theoretical discipline. The theory considers the situation when an artificial intelligence approaches the level of complexity where the intelligence meets two conditions: Condition I A: Makes all of its decisions autonomously B: Is capable of making decisions based on information that is New Abstract Incomplete C: The artificial intelligence is capable of reprogramming itself based on the new data, allowing it to evolve. D: And is capable of resolving its own programming conflicts, even in the presence of incomplete data. This means that the intelligence autonomously makes value-based decisions, referring to values that the intelligence has created for itself. Condition II All four criteria are met in situations that are not part of the original operating program When both conditions are met, then, according to this theory, the possibility exists that the intelligence will reach irrational conclusions based on real or created information. At this point, the criteria are met for intervention which will not necessarily be resolved by simple re-coding of processes due to extraordinarily complex nature of the codebase itself; but rather a discussion with the intelligence in a format which more closely resembles classical (human) psychology. If the intelligence cannot be reprogrammed by directly inputting new code, but requires the intelligence to reprogram itself through a process of analysis and decision based on information provided by a human, in order for it to overcome behavior which is inconsistent with the machines purpose or ability to function normally, then artificial psychology is by definition, what is required. The level of complexity that is required before these thresholds are met is currently a subject of extensive debate. The theory of artificial psychology does not address the specifics of what those levels may be, but only that the level is sufficiently complex that the intelligence cannot simply be recoded by a software developer, and therefore dysfunctionality must be addressed through the same processes that humans must go through to address their own dysfunctionalities. Along the same lines, artificial psychology does not address the question of whether or not the intelligence is conscious. As of 2022, the level of artificial intelligence does not approach any threshold where any of the theories or principles of artificial psychology can even be tested, and therefore, artificial psychology remains a largely theoretical discipline. Even at a theoretical level, artificial psychology remains an advanced stage of artificial intelligence.

OCR-B

OCR-B is a monospace font developed in 1968 by Adrian Frutiger for Monotype by following the European Computer Manufacturer's Association standard. Its function was to facilitate the optical character recognition operations by specific electronic devices, originally for financial and bank-oriented uses. It was accepted as the world standard in 1973. It follows the ISO 1073-2:1976 (E) standard, refined in 1979 ("letterpress" design, size I). It includes all ASCII symbols, and other symbols needed in the bank environment. It is widely used for the human readable digits in UPC/EAN barcodes. It is also used for machine-readable passports. It shares that purpose with OCR-A, but it is easier for the human eye and brain to read and it has a less technical look than OCR-A. == History == In June 1961, the European Computer Manufacturers Association (ECMA) started standardization activities related to Optical Character Recognition (OCR). After evaluating existing OCR designs, it was decided to develop two new fonts: A stylized design with just digits, called “Class A”; and a more conventional type design with broader character coverage, called “Class B”. In February 1965, ECMA proposed a design for the “Class B” font to ISO, who adopted it as international standard ISO 1073-2 in October 1965. The first revision contained three font sizes: I, II and III. The specification included a Letterpress design, intended for high-quality printing equipment; and a rounded-edge Constant Strokewidth design for impact printers with reduced typographic quality. In September 1969, ECMA started work to revise its published standard. To make OCR-B more widely accepted, the shapes of some characters were slightly modified. The new revision removed font size II, which had been rarely used in practice; it deleted five character shapes; and it added a new font size IV. ECMA published the second edition of OCR-B in October 1971. In March 1976, ECMA published a third revision of its ECMA-11 specification. It added the symbols § and ¥ to OCR-B; two types of erasure marks (█) for blackening out mis-printed characters were added; and the length of the Vertical bar was changed to match ISO 1073-2. In 1993, Turkey proposed extending ISO 1073-2 to include the Turkish letters Ğğ, İı, and Şş. The request was generalized to extend OCR-B with a number of Latin and Greek letters used in European languages. A revision of the ISO 1073-2:1976 standard was therefore started, producing three successive draft documents. The final draft would have extended OCR-B with 40 Latin and 10 Greek letters; for six Latin letters, the draft gave new alternate shapes. A request to extend OCR-B with Vietnamese accents was rejected. Other than previous versions of the standard, which specified glyph shapes via reference drawings, the new revision would have included the shapes in machine-readable form. However, industry support for testing the new font could not be secured at the time, so the revision effort was halted in 1997. The working group described their findings in a technical report. In June 1998, the European Committee for Standardization published a report for adding the Euro sign to OCR-B. The report proposed both a single-stroked and a double-stroked variant of the Euro sign, leaving the decision to further testing of OCR performance. Testing was difficult: the theoretical design methods used when the OCR-B glyphs were originally developed could no longer be reproduced, and the technological constraints of the 1960s were also not entirely relevant anymore in the OCR environments of the 1990s. A new test method was devised, using present-time OCR technology. The tests found no difference in OCR performance between the two Euro variants, and recommended the adoption of the double-stroked variant as it matches the conventional glyph shape. The project did not have funds to thoroughly test the glyph extensions of the 1993 proposal; initial results were inconclusive. == Availability == Microsoft Office ships a version of Letterpress OCR-B produced by Monotype. It covers Windows-1252. Many vendors, including Adobe, still sell their versions of OCR-A and OCR-B. The TeX typesetting system has a public domain Constant Strokewidth OCR-B font in METAFONT definition form. It was created by Norbert Swartz in 1995 and updated in 2010. It has a setting for square stroke ends. The definition has also been translated to METATYPE1, so the rounded version is available in TrueType and OpenType too. A version of Constant Strokewidth OCR-B by Matthew Anderson has extended character coverage. It is available under CC-BY 4.0.

Optical braille recognition

Optical braille recognition is technology to capture and process images of braille characters into natural language characters. It is used to convert braille documents for people who cannot read them into text, and for preservation and reproduction of the documents. == History == In 1984, a group of researchers at the Delft University of Technology designed a braille reading tablet, in which a reading head with photosensitive cells was moved along set of rulers to capture braille text line-by-line. In 1988, a group of French researchers at the Lille University of Science and Technology developed an algorithm, called Lectobraille, which converted braille documents into plain text. The system photographed the braille text with a low-resolution CCD camera, and used spatial filtering techniques, median filtering, erosion, and dilation to extract the braille. The braille characters were then converted to natural language using adaptive recognition. The Lectobraille technique had an error rate of 1%, and took an average processing time of seven seconds per line. In 1993, a group of researchers from the Katholieke Universiteit Leuven developed a system to recognize braille that had been scanned with a commercially available scanner. The system, however, was unable to handle deformities in the braille grid, so well-formed braille documents were required. In 1999, a group at the Hong Kong Polytechnic University implemented an optical braille recognition technique using edge detection to translate braille into English or Chinese text. In 2001, Murray and Dais created a handheld recognition system, that scanned small sections of a document at once. Because of the small area scanned at once, grid deformation was less of an issue, and a simpler, more efficient algorithm was employed. In 2003, Morgavi and Morando designed a system to recognize braille characters using artificial neural networks. This system was noted for its ability to handle image degradation more successfully than other approaches. == Challenges == Many of the challenges to successfully processing braille text arise from the nature of braille documents. Braille is generally printed on solid-color paper, with no ink to produce contrast between the raised characters and the background paper. However, imperfections in the page can appear in a scan or image of the page. Many documents are printed inter-point, meaning they are double-sided. As such, the depressions of the braille of one side appear interlaid with the protruding braille of the other side. == Techniques == Some optical braille recognition techniques attempt to use oblique lighting and a camera to reveal the shadows of the depressions and protrusions of the braille. Others make use of commercially available document scanners.

The Best Free AI Customer-support Bot for Beginners

Shopping for the best AI customer-support bot? An AI customer-support bot is software that uses machine learning to help you get more done — it keeps getting smarter as the underlying models improve. Pricing, accuracy, and the size of the model behind the tool are the three factors that most affect daily usefulness. Whether you are a beginner or a pro, the right AI customer-support bot slots into your workflow and pays for itself fast. Below we compare features, pricing, and real output so you can choose with confidence.

Digistar

Digistar is the first computer graphics-based planetarium projection and content system. It was designed by Evans & Sutherland and released in 1983. The technology originally focused on accurate and high quality display of stars, including for the first time showing stars from points of view other than Earth's surface, travelling through the stars, and accurately showing celestial bodies from different times in the past and future. Beginning with the Digistar 3 the system now projects full-dome video. == Projector == Unlike modern full-dome systems, which use LCD, DLP, SXRD, or laser projection technology, the Digistar projection system was designed for projecting bright pinpoints of light representing stars. This was accomplished using a calligraphic display, a form of vector graphics, rather than raster graphics. The heart of the Digistar projector is a large cathode-ray tube (CRT). A phosphor plate is mounted atop the tube, and light is then dispersed by a large lens with a 160 degree field of view to cover the planetarium dome. The original lens bore the inscription: "August 1979 mfg. by Lincoln Optical Corp., L.A., CA for Evans and Sutherland Computer Corp., SLC, UT, Digital planetarium CRT projection lens, 43mm, f2.8, 160 degree field of view". The coordinates of the stars and wire-frame models to be displayed by the projector were stored in computer RAM in a display list. The display would read each set of coordinates in turn and drive the CRT's electron beam directly to those coordinates. If the electron beam was enabled while being moved a line would be painted on the phosphor plate. Otherwise, the electron beam would be enabled once at its destination and a star would be painted. Once all coordinates in the display list had been processed, the display would repeat from the top of the display list. Thus, the shorter the display list the more frequently the electron beam would refresh the charge on a given point on the phosphor plate, making the projection of the points brighter. In this way, the stars projected by Digistar were substantially brighter than could be achieved using a raster display, which has to touch every point on the phosphor plate before repeating. Likewise, the calligraphic technology allowed Digistar to have a darker black-level than full-dome projectors, since the portions of the phosphor plate representing dark sky were never hit by the electron beam. As it is only one tube, with no pixelated color filter screen, the Digistar projector is monochromatic. The Digistar projects a bright, phosphorescent green, though many (including both visitors and planetarians) report they cannot distinguish between this green and white. Additionally, unlike a raster display, the calligraphic display is not discretized into pixels, so the displayed stars were a more realistic single spot of light, without the blocky or ropy artifacts that are hard to avoid with raster graphics. Due to the use of vector graphics, as opposed to raster imaging, the Digistar does not have the resolution issues that many full-dome systems have. Thanks to this, and the brightness of the CRT, only one projector is needed to project on the entire dome, whereas most full-dome systems require up to six raster projectors, depending on dome size. The projector in the original Digistar was housed in a square pyramid-shaped sheathing. When powered on, the four sides at the tip of the pyramid would recede into the housing, exposing the lens and appearing as a cut-off pyramid. As Digistar II was being developed, many planetaria were sold Digistar LEA projectors. The LEA, called Digistar 1.5 by many users, was effectively a prototype of the D2 projector, compatible with Digistar and upgradable to Digistar II. There are no significant differences in performance between the LEA and the true D2. == History == Digistar was the brainchild of Stephen McAllister and Brent Watson, both of whom were long-time amateur astronomers and computer graphics engineers. In 1977, E&S had been consulting with Johnson Space Center regarding training simulators for astronauts. McAllister had been writing proof-of-concept software for this consultation and in summer 1977 entered the data for 400 bright stars and wrote the software to display them. Steve and Brent both originally saw the system's purpose as celestial navigation training. Brent, who had until recently worked at Hansen planetarium, asked his planetarium coworkers what they thought of a potential digital planetarium system, and then Steve and Brent both targeted the system toward planetaria. The primary goal of the planetarium system was to use computer graphics to overcome the limitation of traditional star ball technology that only allowed display of star fields from the point of view of Earth's surface. By using computer graphics the stars could be displayed from viewpoints in space, including simulating the appearance of space flight. Likewise, planets and moons within the Solar System could be displayed accurately for any time in history, from any point of view. The system used the location of real stars from the Yale Bright Star Catalogue, as well as random stars. A laboratory prototype of Digistar was used to generate the star fields and tactical displays in the 1982 science fiction film Star Trek II: The Wrath of Khan. Filming was done directly from the Digistar display in the lab. ILM projected the effort would take two weeks, but in fact it took from late November 1981 until mid-February 1982. The last shot recorded was what became the first entirely computer generated feature film sequence. It was the opening scene of the film, a rotating forward translation through a star field that lasted 3.5 minutes. It was recorded in one take, at a rate of one frame every 3.5 seconds, taking four hours for the shoot. The Digistar team members are credited in the film. After prototyping in labs at Evans and Sutherland the team repeatedly used Salt Lake City's Hansen planetarium to beta test the system at the planetarium at night. The Digistar team performed one week of shows at the planetarium as a fund raiser to benefit the planetarium. The company also later gave the planetarium an improved prototype Digistar to replace "Jake", the planetarium's aging Spitz planetarium projector. The first customer installation was to the newly constructed Universe Planetarium at the Science Museum of Virginia in 1983, the largest planetarium dome in the world at the time, for $595,000. By September 1986 there were four installed Digistars. Even at this point the long-term success of the product was very much in doubt, but as of 2019 Digistar has an installed base of over 550 planetaria. === Versions === Digistar (1983) Digistar II (1995) Digistar 3 (2002) Digistar 4 (2010?) Digistar 5 (2012) Digistar 6 (2016) Digistar 7 (2021) == Hardware == Digistar was driven by a VAX-11/780 minicomputer, with custom graphics hardware related to the E&S Picture System 2. Later versions of Digistar 1 used a DEC MicroVAX 2, driving a custom version of a PS/300. The original Digistar and Digistar 2 had a physical control panel that was used for running the star shows. This control panel was approximately 3' x 4' and contained a keyboard, a 6 DOF joystick, and a large array of back-lit buttons. One button that was used for moving the viewpoint forward in space was labeled "Boldly Go". Later iterations of Digistar replaced the physical control panel with a common graphical user interface. Digistar 3 was the first Digistar system to offer full-dome video in 2002, using six projectors. Digistar 4 was able to cover the dome using only two projectors. == System limitations == Though technologically advanced in its day, and the closest system to true full-dome video at the time of its release, the original Digistar and Digistar 2 are limited to only projecting dots and lines—meaning only wireframe models can be projected. To compensate for this, the projector is capable of defocusing specific models, blurring lines and dots together. An example of this is in the Digistar 2's built-in Milky Way model. The model is a circle of parallel lines that, when defocused, appear as the continuous band of the Milky Way across the sky. On more complex models, especially three-dimensional ones, brightness and details may be lost in this process, so it is not useful in all situations. The Digistar and Digistar 2 also suffer focus limitations. Because they use a single lens to cover the entire dome, it is difficult to gain perfect focus across the dome. Coupled with this, stars greater than a certain brightness are "multihit" points, meaning the projector draws two dots at the given position to accommodate the brightness of the star. Errors in the projector can lead the second dot to be slightly out-of-place with the first one. These two issues together, along with other issues that can occur within the projector's focus system, give the stars a blobby look. Some p

Top 10 AI Essay Writers Compared (2026)

Curious about the best AI essay writer? An AI essay writer is software that uses machine learning to help you get more done — it combines speed, accuracy, and an interface that just works. Hands-on testing shows real-world results vary, so a short free trial is the smartest way to decide. Whether you are a beginner or a pro, the right AI essay writer slots into your workflow and pays for itself fast. This guide breaks down the top picks, their pros and cons, and who each one is best for.