Barney Pell (born March 18, 1968) is an American entrepreneur, angel investor and computer scientist. He was co-founder and CEO of Powerset, a pioneering natural language search startup, search strategist and architect for Microsoft's Bing search engine, a pioneer in the field of general game playing in artificial intelligence, and the architect of the first intelligent agent to fly onboard and control a spacecraft. He was co-founder, Vice Chairman and Chief Strategy Officer of Moon Express; co-founder and chairman of LocoMobi; and Associate Founder of Singularity University. == Career == === Education === Pell received his Bachelor of Science degree in symbolic systems from Stanford University in 1989, where he graduated Phi Beta Kappa and was a National Merit Scholar. Pell earned a PhD in computer science from Cambridge University in 1993, supervised by Stephen Pulman, where he was a Marshall Scholar. === Research === Pell's research is focused on basic problems in the study of intelligence, computer game playing, machine learning, natural language processing, autonomous robotics, and web search. Barney Pell has published over 30 technical papers on topics related to information retrieval, knowledge management, machine learning, artificial intelligence, and scheduling systems. In computer game playing and machine learning, he was a pioneer in the field of General Game Playing, and created programs to generate the rules of chess-like games and programs to play individual games directly from the rules without human assistance. He also did early work on machine learning in the game of Go and on an architecture for pragmatic reasoning for bidding in the game of Bridge. In natural language processing, he was a scientist in the Artificial Intelligence Center at SRI International, where he worked on the Core Language Engine. Barney Pell was the Technical Area Manager of the Collaborative and Assistant Systems area within the Computational Sciences Division (now the Intelligent Systems Division) at NASA Ames Research Center, where he oversaw a staff of 80 scientists working on information retrieval, search, knowledge management, machine learning, semantic technology, human centered systems, collaboration technology, adaptive user interfaces, human robot interaction, and other areas of artificial intelligence. From 1993 to 1998, Barney Pell worked as a Principal Investigator and Senior Computer Scientist at NASA Ames, where he conducted advanced research and development of autonomous control software for NASA's deep space missions. He was the Architect for the Deep Space One Remote Agent Experiment and the Project Lead for the Executive component of the Remote Agent Experiment, the first intelligent agent to fly onboard and control a spacecraft. === Business === Pell is an entrepreneur who has founded or co-founded several business ventures, including Powerset, Moon Express, and LocoMobi. He was the founder and CEO of Powerset, a San Francisco startup company that built a search engine based on natural language processing technology originally developed at XEROX PARC. On May 11, 2008, the company unveiled a tool for searching a fixed subset of Wikipedia using conversational phrases rather than keywords. On July 1, 2008, Microsoft signed an agreement to acquire Powerset for an estimated $100 million. Powerset became a part of Microsoft's search engine, Bing. From 2008 until August 2011, Pell served as Partner, Search Strategist, and Evangelist for Microsoft's search engine, Bing and as Head of Bing's Local and Mobile Search teams. Prior to joining Powerset, Pell was an Entrepreneur-in-Residence at Mayfield Fund, a venture capital firm in Silicon Valley. Pell is also a founder of Moon Express, Inc., a U.S. company awarded a $10M commercial lunar contract by NASA and a competitor in the Google Lunar X PRIZE. Pell was also co-founder and chairman of LocoMobi, Inc., a U.S. company developing mobile, software and hardware technology solutions for the parking industry. LocoMobi was winner of the Tie50 Award in 2014. Pell is also an associate founder of Singularity University and a Machine Learning Fellow at the Creative Destruction Lab at the Rotman School of Management From 1998 to 2000, Pell served as chief strategist and vice president of business development at StockMaster.com (acquired by Red Herring in March, 2000). From 2000 to 2002, Pell was Chief Strategist and Vice President of Business Development for Whizbang Labs. Pell has been an angel investor and advisor to numerous startup companies, including Pulse.io (acquired by Google), Aardvark (acquired by Google), Appjet (acquired by Google), Jibe Mobile (acquired by Google), Movity (acquired by Trulia), QuestBridge, BrandYourself, CrowdFlower (acquired by Appen), and LinkedIn. === Views and predictions === Pell has expressed views and predictions regarding technological advancements in coming years. He believes that humans will soon have "brain-machine interfaces that will let people interact with each other as if they had 'hangouts' in their mind." Pell predicts these interfaces to become available within 20 to 30 years. Pell also predicts advancements in bodily augmentation, such as "even-better-than-human prosthetics and high-quality tissue engineering within 10 years." Pell believes that with advancements in space exploration technology the moon will soon be a commercially viable resource for material such as platinum and water. == Awards and recognition == In 1986, Pell was awarded a National Merit Scholarship. In 1989, Pell was awarded a Marshall Scholarship. In 1989, Pell was elected Phi Beta Kappa. In 1997, Pell was part of the team award a NASA Software of the Year Award for the Deep Space 1 Remote Agent.
Image texture
An image texture is the small-scale structure perceived on an image, based on the spatial arrangement of color or intensities. It can be quantified by a set of metrics calculated in image processing. Image texture metrics give us information about the whole image or selected regions. Image textures can be artificially created or found in natural scenes captured in an image. Image textures are one way that can be used to help in segmentation or classification of images. For more accurate segmentation the most useful features are spatial frequency and an average grey level. To analyze an image texture in computer graphics, there are two ways to approach the issue: structured approach and statistical approach. == Structured approach == A structured approach sees an image texture as a set of primitive texels in some regular or repeated pattern. This works well when analyzing artificial textures. To obtain a structured description a characterization of the spatial relationship of the texels is gathered by using Voronoi tessellation of the texels. == Statistical approach == A statistical approach sees an image texture as a quantitative measure of the arrangement of intensities in a region. In general this approach is easier to compute and is more widely used, since natural textures are made of patterns of irregular subelements. === Edge detection === The use of edge detection is to determine the number of edge pixels in a specified region, helps determine a characteristic of texture complexity. After edges have been found the direction of the edges can also be applied as a characteristic of texture and can be useful in determining patterns in the texture. These directions can be represented as an average or in a histogram. Consider a region with N pixels. the gradient-based edge detector is applied to this region by producing two outputs for each pixel p: the gradient magnitude Mag(p) and the gradient direction Dir(p). The edgeness per unit area can be defined by F e d g e n e s s = | { p | M a g ( p ) > T } | N {\displaystyle F_{edgeness}={\frac {|\{p|Mag(p)>T\}|}{N}}} for some threshold T. To include orientation with edgeness histograms for both gradient magnitude and gradient direction can be used. Hmag(R) denotes the normalized histogram of gradient magnitudes of region R, and Hdir(R) denotes the normalized histogram of gradient orientations of region R. Both are normalized according to the size NR Then F m a g , d i r = ( H m a g ( R ) , H d i r ( R ) ) {\displaystyle F_{mag,dir}=(H_{mag}(R),H_{dir}(R))} is a quantitative texture description of region R. === Co-occurrence matrices === The co-occurrence matrix captures numerical features of a texture using spatial relations of similar gray tones. Numerical features computed from the co-occurrence matrix can be used to represent, compare, and classify textures. The following are a subset of standard features derivable from a normalized co-occurrence matrix: A n g u l a r 2 n d M o m e n t = ∑ i ∑ j p [ i , j ] 2 C o n t r a s t = ∑ i = 1 N g ∑ j = 1 N g n 2 p [ i , j ] , where | i − j | = n C o r r e l a t i o n = ∑ i = 1 N g ∑ j = 1 N g ( i j ) p [ i , j ] − μ x μ y σ x σ y E n t r o p y = − ∑ i ∑ j p [ i , j ] l n ( p [ i , j ] ) {\displaystyle {\begin{aligned}Angular{\text{ }}2nd{\text{ }}Moment&=\sum _{i}\sum _{j}p[i,j]^{2}\\Contrast&=\sum _{i=1}^{Ng}\sum _{j=1}^{Ng}n^{2}p[i,j]{\text{, where }}|i-j|=n\\Correlation&={\frac {\sum _{i=1}^{Ng}\sum _{j=1}^{Ng}(ij)p[i,j]-\mu _{x}\mu _{y}}{\sigma _{x}\sigma _{y}}}\\Entropy&=-\sum _{i}\sum _{j}p[i,j]ln(p[i,j])\\\end{aligned}}} where p [ i , j ] {\displaystyle p[i,j]} is the [ i , j ] {\displaystyle [i,j]} th entry in a gray-tone spatial dependence matrix, and Ng is the number of distinct gray-levels in the quantized image. One negative aspect of the co-occurrence matrix is that the extracted features do not necessarily correspond to visual perception. It is used in dentistry for the objective evaluation of lesions [DOI: 10.1155/2020/8831161], treatment efficacy [DOI: 10.3390/ma13163614; DOI: 10.11607/jomi.5686; DOI: 10.3390/ma13173854; DOI: 10.3390/ma13132935] and bone reconstruction during healing [DOI: 10.5114/aoms.2013.33557; DOI: 10.1259/dmfr/22185098; EID: 2-s2.0-81455161223; DOI: 10.3390/ma13163649]. === Laws texture energy measures === Another approach is to use local masks to detect various types of texture features. Laws originally used four vectors representing texture features to create sixteen 2D masks from the outer products of the pairs of vectors. The four vectors and relevant features were as follows: L5 = [ +1 +4 6 +4 +1 ] (Level) E5 = [ -1 -2 0 +2 +1 ] (Edge) S5 = [ -1 0 2 0 -1 ] (Spot) R5 = [ +1 -4 6 -4 +1 ] (Ripple) To these 4, a fifth is sometimes added: W5 = [ -1 +2 0 -2 +1 ] (Wave) From Laws' 4 vectors, 16 5x5 "energy maps" are then filtered down to 9 in order to remove certain symmetric pairs. For instance, L5E5 measures vertical edge content and E5L5 measures horizontal edge content. The average of these two measures is the "edginess" of the content. The resulting 9 maps used by Laws are as follows: L5E5/E5L5 L5R5/R5L5 E5S5/S5E5 S5S5 R5R5 L5S5/S5L5 E5E5 E5R5/R5E5 S5R5/R5S5 Running each of these nine maps over an image to create a new image of the value of the origin ([2,2]) results in 9 "energy maps," or conceptually an image with each pixel associated with a vector of 9 texture attributes. === Autocorrelation and power spectrum === The autocorrelation function of an image can be used to detect repetitive patterns of textures. == Texture segmentation == The use of image texture can be used as a description for regions into segments. There are two main types of segmentation based on image texture, region based and boundary based. Though image texture is not a perfect measure for segmentation it is used along with other measures, such as color, that helps solve segmenting in image. === Region based === Attempts to group or cluster pixels based on texture properties. === Boundary based === Attempts to group or cluster pixels based on edges between pixels that come from different texture properties.
Monica S. Lam
Monica Sin-Ling Lam is an American computer scientist. She is a professor in the Computer Science Department at Stanford University. == Education == Monica Lam received a B.Sc. from University of British Columbia in 1980 and a Ph.D. in computer science from Carnegie Mellon University in 1987. == Career == Lam joined the faculty of Computer Science at Stanford University in 1988. She has contributed to the research of a wide range of computer systems topics including compilers, program analysis, operating systems, security, computer architecture, and high-performance computing. More recently, she is working in natural language processing, and virtual assistants with an emphasis on privacy protection. She is the faculty director of the Open Virtual Assistant Lab, which organized the first workshop for the World Wide Voice Web. The lab developed the open-source Almond voice assistant, which is sponsored by the National Science Foundation. Almond received Popular Science's Best of What's New award in 2019. Previously, Lam led the SUIF (Stanford University Intermediate Format) Compiler project, which produced a widely used compiler infrastructure known for its locality optimizations and interprocedural parallelization. Many of the compiler techniques she developed have been adopted by industry. Her other research projects included the architecture and compiler for the CMU Warp machine, a systolic array of VLIW processors, and the Stanford DASH distributed shared memory machine. In 1998, she took a sabbatical leave from Stanford to help start Tensilica Inc., a company that specializes in configurable processor cores. In another research project, her program analysis group developed a collection of tools for improving software security and reliability. They developed the first scalable context-sensitive inclusion-based pointer analysis and a freely available tool called BDDBDDB, that allows programmers to express context-sensitive analyses simply by writing Datalog queries. Other tools developed include Griffin, static and dynamic analysis for finding security vulnerabilities in Web applications such as SQL injection, a static and dynamic program query language called QL, a static memory leak detector called Clouseau, a dynamic buffer overrun detector called CRED, and a dynamic error diagnosis tool called DIDUCE. In the Collective project, her research group and she developed the concept of a livePC: subscribers of the livePC will automatically run the latest of the published PC virtual images with each reboot. This approach allows computers to be managed scalably and securely. In 2005, the group started a company called MokaFive to transfer the technology to industry. She also directed the MobiSocial laboratory at Stanford, as part of the Programmable Open Mobile Internet 2020 initiative. Lam is also the cofounder of Omlet, which launched in 2014. Omlet is the first product from MobiSocial. Omlet is an open, decentralized social networking tool, based on an extensible chat platform. Lam chaired the ACM SIGPLAN Programming Languages Design and Implementation Conference in 2000, served on the Editorial Board of ACM Transactions on Computer Systems and numerous program committees for conferences on languages and compilers (PLDI, POPL), operating systems (SOSP), and computer architecture (ASPLOS, ISCA). == Awards and honors == National Academy of Engineering member, 2019 University of British Columbia Computer Science 50th Anniversary Research Award, 2018 Fellow of the ACM, 2007 ACM Programming Language Design and Implementation Best Paper Award in 2004 ACM SIGSOFT Distinguished Paper Award in 2002 ACM Most Influential Programming Language Design and Implementation Paper Award in 2001 NSF Young Investigator award in 1992 Two of her papers were recognized in "20 Years of PLDI--a Selection (1979-1999)" One of her papers was recognized in the "25 Years of the International Symposia on Computer Architecture", 1988. == Selected works == Compilers: Principles, Techniques and Tools (2d Ed) (2006) (the "Dragon Book") by Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman (ISBN 0-321-48681-1) A Systolic Array Optimizing Compiler (1989) (ISBN 0-89838-300-5) Monica Lam, Dissertation
Moore machine
In the theory of computation, a Moore machine is a finite-state machine whose current output values are determined only by its current state. This is in contrast to a Mealy machine, whose output values are determined both by its current state and by the values of its inputs. Like other finite state machines, in Moore machines, the input typically influences the next state. Thus the input may indirectly influence subsequent outputs, but not the current or immediate output. The Moore machine is named after Edward F. Moore, who presented the concept in a 1956 paper, “Gedanken-experiments on Sequential Machines.” == Formal definition == A Moore machine can be defined as a 6-tuple ( S , s 0 , Σ , Λ , δ , G ) {\displaystyle (S,s_{0},\Sigma ,\Lambda ,\delta ,G)} consisting of the following: A finite set of states S {\displaystyle S} A start state (also called initial state) s 0 {\displaystyle s_{0}} which is an element of S {\displaystyle S} A finite set called the input alphabet Σ {\displaystyle \Sigma } A finite set called the output alphabet Λ {\displaystyle \Lambda } A transition function δ : S × Σ → S {\displaystyle \delta :S\times \Sigma \rightarrow S} mapping a state and the input alphabet to the next state An output function G : S → Λ {\displaystyle G:S\rightarrow \Lambda } mapping each state to the output alphabet "Evolution across time" is realized in this abstraction by having the state machine consult the time-changing input symbol at discrete "timer ticks" t 0 , t 1 , t 2 , . . . {\displaystyle t_{0},t_{1},t_{2},...} and react according to its internal configuration at those idealized instants, or else having the state machine wait for a next input symbol (as on a FIFO) and react whenever it arrives. A Moore machine can be regarded as a restricted type of finite-state transducer. == Visual representation == === Table === A state transition table is a table listing all the triples in the transition relation δ : S × Σ → S {\displaystyle \delta :S\times \Sigma \rightarrow S} . === Diagram === The state diagram for a Moore machine, or Moore diagram, is a state diagram that associates an output value with each state. == Relationship with Mealy machines == As Moore and Mealy machines are both types of finite-state machines, they are equally expressive: either type can be used to parse a regular language. The difference between Moore machines and Mealy machines is that in the latter, the output of a transition is determined by the combination of current state and current input ( S × Σ {\displaystyle S\times \Sigma } as the domain of G {\displaystyle G} ), as opposed to just the current state ( S {\displaystyle S} as the domain of G {\displaystyle G} ). When represented as a state diagram, for a Moore machine, each node (state) is labeled with an output value; for a Mealy machine, each arc (transition) is labeled with an output value. Every Moore machine M {\displaystyle M} is equivalent to the Mealy machine with the same states and transitions and the output function G ( s , σ ) = G M ( δ M ( s , σ ) ) {\displaystyle G(s,\sigma )=G_{M}(\delta _{M}(s,\sigma ))} , which takes each state-input pair ( s , σ ) {\displaystyle (s,\sigma )} and yields G M ( δ M ( s , σ ) ) {\displaystyle G_{M}(\delta _{M}(s,\sigma ))} , where G M {\displaystyle G_{M}} is M {\displaystyle M} 's output function and δ M {\displaystyle \delta _{M}} is M {\displaystyle M} 's transition function. However, not every Mealy machine can be converted to an equivalent Moore machine. Some can be converted only to an almost equivalent Moore machine, with outputs shifted in time. This is due to the way that state labels are paired with transition labels to form the input/output pairs. Consider a transition s i → s j {\displaystyle s_{i}\rightarrow s_{j}} from state s i {\displaystyle s_{i}} to state s j {\displaystyle s_{j}} . The input causing the transition s i → s j {\displaystyle s_{i}\rightarrow s_{j}} labels the edge ( s i , s j ) {\displaystyle (s_{i},s_{j})} . The output corresponding to that input, is the label of state s i {\displaystyle s_{i}} . Notice that this is the source state of the transition. So for each input, the output is already fixed before the input is received, and depends solely on the present state. This is the original definition by E. Moore. It is a common mistake to use the label of state s j {\displaystyle s_{j}} as output for the transition s i → s j {\displaystyle s_{i}\rightarrow s_{j}} . == Examples == Types according to number of inputs/outputs. === Simple === Simple Moore machines have one input and one output: edge detector using XOR binary adding machine clocked sequential systems (a restricted form of Moore machine where the state changes only when the global clock signal changes) Most digital electronic systems are designed as clocked sequential systems. Clocked sequential systems are a restricted form of Moore machine where the state changes only when the global clock signal changes. Typically the current state is stored in flip-flops, and a global clock signal is connected to the "clock" input of the flip-flops. Clocked sequential systems are one way to solve metastability problems. A typical electronic Moore machine includes a combinational logic chain to decode the current state into the outputs (lambda). The instant the current state changes, those changes ripple through that chain, and almost instantaneously the output gets updated. There are design techniques to ensure that no glitches occur on the outputs during that brief period while those changes are rippling through the chain, but most systems are designed so that glitches during that brief transition time are ignored or are irrelevant. The outputs then stay the same indefinitely (LEDs stay bright, power stays connected to the motors, solenoids stay energized, etc.), until the Moore machine changes state again. ==== Worked example ==== A sequential network has one input and one output. The output becomes 1 and remains 1 thereafter when at least two 0's and two 1's have occurred as inputs. A Moore machine with nine states for the above description is shown on the right. The initial state is state A, and the final state is state I. The state table for this example is as follows: === Complex === More complex Moore machines can have multiple inputs as well as multiple outputs. == Gedanken-experiments == In Moore's 1956 paper "Gedanken-experiments on Sequential Machines", the ( n ; m ; p ) {\displaystyle (n;m;p)} automata (or machines) S {\displaystyle S} are defined as having n {\displaystyle n} states, m {\displaystyle m} input symbols and p {\displaystyle p} output symbols. Nine theorems are proved about the structure of S {\displaystyle S} , and experiments with S {\displaystyle S} . Later, " S {\displaystyle S} machines" became known as "Moore machines". At the end of the paper, in Section "Further problems", the following task is stated: Another directly following problem is the improvement of the bounds given at the theorems 8 and 9. Moore's Theorem 8 is formulated as: Given an arbitrary ( n ; m ; p ) {\displaystyle (n;m;p)} machine S {\displaystyle S} , such that every two of its states are distinguishable from one another, then there exists an experiment of length n ( n − 1 ) 2 {\displaystyle {\tfrac {n(n-1)}{2}}} which determines the state of S {\displaystyle S} at the end of the experiment. In 1957, A. A. Karatsuba proved the following two theorems, which completely solved Moore's problem on the improvement of the bounds of the experiment length of his "Theorem 8". Theorem A. If S {\displaystyle S} is an ( n ; m ; p ) {\displaystyle (n;m;p)} machine, such that every two of its states are distinguishable from one another, then there exists a branched experiment of length at most ( n − 1 ) ( n − 2 ) 2 + 1 {\displaystyle {\tfrac {(n-1)(n-2)}{2}}+1} through which one may determine the state of S {\displaystyle S} at the end of the experiment. Theorem B. There exists an ( n ; m ; p ) {\displaystyle (n;m;p)} machine, every two states of which are distinguishable from one another, such that the length of the shortest experiments establishing the state of the machine at the end of the experiment is equal to ( n − 1 ) ( n − 2 ) 2 + 1 {\displaystyle {\tfrac {(n-1)(n-2)}{2}}+1} . Theorems A and B were used for the basis of the course work of a student of the fourth year, A. A. Karatsuba, "On a problem from the automata theory", which was distinguished by testimonial reference at the competition of student works of the faculty of mechanics and mathematics of Moscow State University in 1958. The paper by Karatsuba was given to the journal Uspekhi Mat. Nauk on 17 December 1958 and was published there in June 1960. Until the present day (2011), Karatsuba's result on the length of experiments is the only exact nonlinear result, both in automata theory, and in similar problems of computational complexity theory.
The Best Free AI Photo Editor for Beginners
Comparing the best AI photo editor? An AI photo editor is software that uses machine learning to help you get more done — it lowers the barrier so anyone can produce professional output. Privacy matters too: check whether your data trains the model and whether a no-log or enterprise tier is available. Whether you are a beginner or a pro, the right AI photo editor slots into your workflow and pays for itself fast. We tested the leading options and ranked them by quality, value, and ease of use.
Multi-scale approaches
The scale space representation of a signal obtained by Gaussian smoothing satisfies a number of special properties, scale-space axioms, which make it into a special form of multi-scale representation. There are, however, also other types of "multi-scale approaches" in the areas of computer vision, image processing and signal processing, in particular the notion of wavelets. The purpose of this article is to describe a few of these approaches: == Scale-space theory for one-dimensional signals == For one-dimensional signals, there exists quite a well-developed theory for continuous and discrete kernels that guarantee that new local extrema or zero-crossings cannot be created by a convolution operation. For continuous signals, it holds that all scale-space kernels can be decomposed into the following sets of primitive smoothing kernels: the Gaussian kernel : g ( x , t ) = 1 2 π t exp ( − x 2 / 2 t ) {\displaystyle g(x,t)={\frac {1}{\sqrt {2\pi t}}}\exp({-x^{2}/2t})} where t > 0 {\displaystyle t>0} , truncated exponential kernels (filters with one real pole in the s-plane): h ( x ) = exp ( − a x ) {\displaystyle h(x)=\exp({-ax})} if x ≥ 0 {\displaystyle x\geq 0} and 0 otherwise where a > 0 {\displaystyle a>0} h ( x ) = exp ( b x ) {\displaystyle h(x)=\exp({bx})} if x ≤ 0 {\displaystyle x\leq 0} and 0 otherwise where b > 0 {\displaystyle b>0} , translations, rescalings. For discrete signals, we can, up to trivial translations and rescalings, decompose any discrete scale-space kernel into the following primitive operations: the discrete Gaussian kernel T ( n , t ) = I n ( α t ) {\displaystyle T(n,t)=I_{n}(\alpha t)} where α , t > 0 {\displaystyle \alpha ,t>0} where I n {\displaystyle I_{n}} are the modified Bessel functions of integer order, generalized binomial kernels corresponding to linear smoothing of the form f o u t ( x ) = p f i n ( x ) + q f i n ( x − 1 ) {\displaystyle f_{out}(x)=pf_{in}(x)+qf_{in}(x-1)} where p , q > 0 {\displaystyle p,q>0} f o u t ( x ) = p f i n ( x ) + q f i n ( x + 1 ) {\displaystyle f_{out}(x)=pf_{in}(x)+qf_{in}(x+1)} where p , q > 0 {\displaystyle p,q>0} , first-order recursive filters corresponding to linear smoothing of the form f o u t ( x ) = f i n ( x ) + α f o u t ( x − 1 ) {\displaystyle f_{out}(x)=f_{in}(x)+\alpha f_{out}(x-1)} where α > 0 {\displaystyle \alpha >0} f o u t ( x ) = f i n ( x ) + β f o u t ( x + 1 ) {\displaystyle f_{out}(x)=f_{in}(x)+\beta f_{out}(x+1)} where β > 0 {\displaystyle \beta >0} , the one-sided Poisson kernel p ( n , t ) = e − t t n n ! {\displaystyle p(n,t)=e^{-t}{\frac {t^{n}}{n!}}} for n ≥ 0 {\displaystyle n\geq 0} where t ≥ 0 {\displaystyle t\geq 0} p ( n , t ) = e − t t − n ( − n ) ! {\displaystyle p(n,t)=e^{-t}{\frac {t^{-n}}{(-n)!}}} for n ≤ 0 {\displaystyle n\leq 0} where t ≥ 0 {\displaystyle t\geq 0} . From this classification, it is apparent that we require a continuous semi-group structure, there are only three classes of scale-space kernels with a continuous scale parameter; the Gaussian kernel which forms the scale-space of continuous signals, the discrete Gaussian kernel which forms the scale-space of discrete signals and the time-causal Poisson kernel that forms a temporal scale-space over discrete time. If we on the other hand sacrifice the continuous semi-group structure, there are more options: For discrete signals, the use of generalized binomial kernels provides a formal basis for defining the smoothing operation in a pyramid. For temporal data, the one-sided truncated exponential kernels and the first-order recursive filters provide a way to define time-causal scale-spaces that allow for efficient numerical implementation and respect causality over time without access to the future. The first-order recursive filters also provide a framework for defining recursive approximations to the Gaussian kernel that in a weaker sense preserve some of the scale-space properties.
Additive smoothing
In statistics, additive smoothing, also called Laplace smoothing or Lidstone smoothing, is a technique used to smooth count data, eliminating issues caused by certain values having 0 occurrences. Given a set of observation counts x = ⟨ x 1 , x 2 , … , x d ⟩ {\displaystyle \mathbf {x} =\langle x_{1},x_{2},\ldots ,x_{d}\rangle } from a d {\displaystyle d} -dimensional multinomial distribution with N {\displaystyle N} trials, a "smoothed" version of the counts gives the estimator θ ^ i = x i + α N + α d ( i = 1 , … , d ) , {\displaystyle {\hat {\theta }}_{i}={\frac {x_{i}+\alpha }{N+\alpha d}}\qquad (i=1,\ldots ,d),} where the smoothed count x ^ i = N θ ^ i {\displaystyle {\hat {x}}_{i}=N{\hat {\theta }}_{i}} , and the "pseudocount" α > 0 is a smoothing parameter, with α = 0 corresponding to no smoothing (this parameter is explained in § Pseudocount below). Additive smoothing is a type of shrinkage estimator, as the resulting estimate will be between the empirical probability (relative frequency) x i / N {\displaystyle x_{i}/N} and the uniform probability 1 / d . {\displaystyle 1/d.} Common choices for α are 0 (no smoothing), +1⁄2 (the Jeffreys prior), or 1 (Laplace's rule of succession), but the parameter may also be set empirically based on the observed data. From a Bayesian point of view, this corresponds to the expected value of the posterior distribution, using a symmetric Dirichlet distribution with parameter α as a prior distribution. In the special case where the number of categories is 2, this is equivalent to using a beta distribution as the conjugate prior for the parameters of the binomial distribution. == History == Laplace came up with this smoothing technique when he tried to estimate the chance that the sun will rise tomorrow. His rationale was that even given a large sample of days with the rising sun, we still can not be completely sure that the sun will still rise tomorrow (known as the sunrise problem). == Pseudocount == A pseudocount is an amount (not generally an integer, despite its name) added to the number of observed cases in order to change the expected probability in a model of those data, when not known to be zero. It is so named because, roughly speaking, a pseudo-count of value α {\displaystyle \alpha } weighs into the posterior distribution similarly to each category having an additional count of α {\displaystyle \alpha } . If the number of occurrences of each item i {\displaystyle i} is x i {\displaystyle x_{i}} out of N {\displaystyle N} samples, the empirical probability of event i {\displaystyle i} is p i , empirical = x i N , {\displaystyle p_{i,{\text{empirical}}}={\frac {x_{i}}{N}},} but the posterior probability when additively smoothed is p i , α -smoothed = x i + α N + α d , {\displaystyle p_{i,\alpha {\text{-smoothed}}}={\frac {x_{i}+\alpha }{N+\alpha d}},} as if to increase each count x i {\displaystyle x_{i}} by α {\displaystyle \alpha } a priori. Depending on the prior knowledge, which is sometimes a subjective value, a pseudocount may have any non-negative finite value. It may only be zero (or the possibility ignored) if impossible by definition, such as the possibility of a decimal digit of π being a letter, or a physical possibility that would be rejected and so not counted, such as a computer printing a letter when a valid program for π is run, or excluded and not counted because of no interest, such as if only interested in the zeros and ones. Generally, there is also a possibility that no value may be computable or observable in a finite time (see the halting problem). But at least one possibility must have a non-zero pseudocount, otherwise no prediction could be computed before the first observation. The relative values of pseudocounts represent the relative prior expected probabilities of their possibilities. The sum of the pseudocounts, which may be very large, represents the estimated weight of the prior knowledge compared with all the actual observations (one for each) when determining the expected probability. In any observed data set or sample there is the possibility, especially with low-probability events and with small data sets, of a possible event not occurring. Its observed frequency is therefore zero, apparently implying a probability of zero. This oversimplification is inaccurate and often unhelpful, particularly in probability-based machine learning techniques such as artificial neural networks and hidden Markov models. By artificially adjusting the probability of rare (but not impossible) events so those probabilities are not exactly zero, zero-frequency problems are avoided. Also see Cromwell's rule. === Choice of pseudocount === ==== Weakly informative prior ==== One common approach is to add 1 to each observed number of events, including the zero-count possibilities. This is sometimes called Laplace's rule of succession. This approach is equivalent to assuming a uniform prior distribution over the probabilities for each possible event (spanning the simplex where each probability is between 0 and 1, and they all sum to 1). Using the Jeffreys prior approach, a pseudocount of one half should be added to each possible outcome. Pseudocounts should be set to one or one-half only when there is no prior knowledge at all – see the principle of indifference. However, given appropriate prior knowledge, the sum should be adjusted in proportion to the expectation that the prior probabilities should be considered correct, despite evidence to the contrary – see further analysis. Higher values are appropriate inasmuch as there is prior knowledge of the true values (for a mint-condition coin, say); lower values inasmuch as there is prior knowledge that there is probable bias, but of unknown degree (for a bent coin, say). ==== Frequentist interval ==== One way to motivate pseudocounts, particularly for binomial data, is via a formula for the midpoint of an interval estimate, particularly a binomial proportion confidence interval. The best-known is due to Edwin Bidwell Wilson, in Wilson (1927): the midpoint of the Wilson score interval corresponding to z {\displaystyle z} standard deviations on either side is n S + z n + 2 z {\displaystyle {\frac {n_{S}+z}{n+2z}}} Taking z = 2 {\displaystyle z=2} standard deviations to approximate a 95% confidence interval ( z ≈ 1.96 {\displaystyle z\approx 1.96} ) yields pseudocount of 2 for each outcome, so 4 in total, colloquially known as the "plus four rule": n S + 2 n + 4 {\displaystyle {\frac {n_{S}+2}{n+4}}} This is also the midpoint of the Agresti–Coull interval (Agresti & Coull 1998). ==== Known incidence rates ==== Often the bias of an unknown trial population is tested against a control population with known parameters (incidence rates) μ = ⟨ μ 1 , μ 2 , … , μ d ⟩ . {\displaystyle {\boldsymbol {\mu }}=\langle \mu _{1},\mu _{2},\ldots ,\mu _{d}\rangle .} In this case the uniform probability 1 / d {\displaystyle 1/d} should be replaced by the known incidence rate of the control population μ i {\displaystyle \mu _{i}} to calculate the smoothed estimator: θ ^ i = x i + μ i α d N + α d ( i = 1 , … , d ) . {\displaystyle {\hat {\theta }}_{i}={\frac {x_{i}+\mu _{i}\alpha d}{N+\alpha d}}\qquad (i=1,\ldots ,d).} As a consistency check, if the empirical estimator happens to equal the incidence rate, i.e. μ i = x i / N , {\displaystyle \mu _{i}=x_{i}/N,} the smoothed estimator is independent of α {\displaystyle \alpha } and also equals the incidence rate. == Applications == === Classification === Additive smoothing is commonly a component of naive Bayes classifiers. === Statistical language modelling === In a bag of words model of natural language processing and information retrieval, the data consists of the number of occurrences of each word in a document. Additive smoothing allows the assignment of non-zero probabilities to words which do not occur in the sample. Studies have shown that additive smoothing is more effective than other probability smoothing methods in several retrieval tasks such as language-model-based pseudo-relevance feedback and recommender systems.