The generalized distributive law (GDL) is a generalization of the distributive property which gives rise to a general message passing algorithm. It is a synthesis of the work of many authors in the information theory, digital communications, signal processing, statistics, and artificial intelligence communities. The law and algorithm were introduced in a semi-tutorial by Srinivas M. Aji and Robert J. McEliece with the same title. == Introduction == "The distributive law in mathematics is the law relating the operations of multiplication and addition, stated symbolically, a ∗ ( b + c ) = a ∗ b + a ∗ c {\displaystyle a(b+c)=ab+ac} ; that is, the monomial factor a {\displaystyle a} is distributed, or separately applied, to each term of the binomial factor b + c {\displaystyle b+c} , resulting in the product a ∗ b + a ∗ c {\displaystyle ab+ac} " – Britannica. As it can be observed from the definition, application of distributive law to an arithmetic expression reduces the number of operations in it. In the previous example the total number of operations reduced from three (two multiplications and an addition in a ∗ b + a ∗ c {\displaystyle ab+ac} ) to two (one multiplication and one addition in a ∗ ( b + c ) {\displaystyle a(b+c)} ). Generalization of distributive law leads to a large family of fast algorithms. This includes the FFT and Viterbi algorithm. This is explained in a more formal way in the example below: α ( a , b ) = d e f ∑ c , d , e ∈ A f ( a , c , b ) g ( a , d , e ) {\displaystyle \alpha (a,\,b){\stackrel {\mathrm {def} }{=}}\displaystyle \sum \limits _{c,d,e\in A}f(a,\,c,\,b)\,g(a,\,d,\,e)} where f ( ⋅ ) {\displaystyle f(\cdot )} and g ( ⋅ ) {\displaystyle g(\cdot )} are real-valued functions, a , b , c , d , e ∈ A {\displaystyle a,b,c,d,e\in A} and | A | = q {\displaystyle |A|=q} (say) Here we are "marginalizing out" the independent variables ( c {\displaystyle c} , d {\displaystyle d} , and e {\displaystyle e} ) to obtain the result. When we are calculating the computational complexity, we can see that for each q 2 {\displaystyle q^{2}} pairs of ( a , b ) {\displaystyle (a,b)} , there are q 3 {\displaystyle q^{3}} terms due to the triplet ( c , d , e ) {\displaystyle (c,d,e)} which needs to take part in the evaluation of α ( a , b ) {\displaystyle \alpha (a,\,b)} with each step having one addition and one multiplication. Therefore, the total number of computations needed is 2 ⋅ q 2 ⋅ q 3 = 2 q 5 {\displaystyle 2\cdot q^{2}\cdot q^{3}=2q^{5}} . Hence the asymptotic complexity of the above function is O ( n 5 ) {\displaystyle O(n^{5})} . If we apply the distributive law to the RHS of the equation, we get the following: α ( a , b ) = d e f ∑ c ∈ A f ( a , c , b ) ⋅ ∑ d , e ∈ A g ( a , d , e ) {\displaystyle \alpha (a,\,b){\stackrel {\mathrm {def} }{=}}\displaystyle \sum \limits _{c\in A}f(a,\,c,\,b)\cdot \sum _{d,\,e\in A}g(a,\,d,\,e)} This implies that α ( a , b ) {\displaystyle \alpha (a,\,b)} can be described as a product α 1 ( a , b ) ⋅ α 2 ( a ) {\displaystyle \alpha _{1}(a,\,b)\cdot \alpha _{2}(a)} where α 1 ( a , b ) = d e f ∑ c ∈ A f ( a , c , b ) {\displaystyle \alpha _{1}(a,b){\stackrel {\mathrm {def} }{=}}\displaystyle \sum \limits _{c\in A}f(a,\,c,\,b)} and α 2 ( a ) = d e f ∑ d , e ∈ A g ( a , d , e ) {\displaystyle \alpha _{2}(a){\stackrel {\mathrm {def} }{=}}\displaystyle \sum \limits _{d,\,e\in A}g(a,\,d,\,e)} Now, when we are calculating the computational complexity, we can see that there are q 3 {\displaystyle q^{3}} additions in α 1 ( a , b ) {\displaystyle \alpha _{1}(a,\,b)} and α 2 ( a ) {\displaystyle \alpha _{2}(a)} each and there are q 2 {\displaystyle q^{2}} multiplications when we are using the product α 1 ( a , b ) ⋅ α 2 ( a ) {\displaystyle \alpha _{1}(a,\,b)\cdot \alpha _{2}(a)} to evaluate α ( a , b ) {\displaystyle \alpha (a,\,b)} . Therefore, the total number of computations needed is q 3 + q 3 + q 2 = 2 q 3 + q 2 {\displaystyle q^{3}+q^{3}+q^{2}=2q^{3}+q^{2}} . Hence the asymptotic complexity of calculating α ( a , b ) {\displaystyle \alpha (a,b)} reduces to O ( n 3 ) {\displaystyle O(n^{3})} from O ( n 5 ) {\displaystyle O(n^{5})} . This shows by an example that applying distributive law reduces the computational complexity which is one of the good features of a "fast algorithm". == History == Some of the problems that used distributive law to solve can be grouped as follows: Decoding algorithms: A GDL like algorithm was used by Gallager's for decoding low density parity-check codes. Based on Gallager's work Tanner introduced the Tanner graph and expressed Gallagers work in message passing form. The tanners graph also helped explain the Viterbi algorithm. It is observed by Forney that Viterbi's maximum likelihood decoding of convolutional codes also used algorithms of GDL-like generality. Forward–backward algorithm: The forward backward algorithm helped as an algorithm for tracking the states in the Markov chain. And this also was used the algorithm of GDL like generality Artificial intelligence: The notion of junction trees has been used to solve many problems in AI. Also the concept of bucket elimination used many of the concepts. == The MPF problem == MPF or marginalize a product function is a general computational problem which as special case includes many classical problems such as computation of discrete Hadamard transform, maximum likelihood decoding of a linear code over a memory-less channel, and matrix chain multiplication. The power of the GDL lies in the fact that it applies to situations in which additions and multiplications are generalized. A commutative semiring is a good framework for explaining this behavior. It is defined over a set K {\displaystyle K} with operators " + {\displaystyle +} " and " . {\displaystyle .} " where ( K , + ) {\displaystyle (K,\,+)} and ( K , . ) {\displaystyle (K,\,.)} are a commutative monoids and the distributive law holds. Let p 1 , … , p n {\displaystyle p_{1},\ldots ,p_{n}} be variables such that p 1 ∈ A 1 , … , p n ∈ A n {\displaystyle p_{1}\in A_{1},\ldots ,p_{n}\in A_{n}} where A {\displaystyle A} is a finite set and | A i | = q i {\displaystyle |A_{i}|=q_{i}} . Here i = 1 , … , n {\displaystyle i=1,\ldots ,n} . If S = { i 1 , … , i r } {\displaystyle S=\{i_{1},\ldots ,i_{r}\}} and S ⊂ { 1 , … , n } {\displaystyle S\,\subset \{1,\ldots ,n\}} , let A S = A i 1 × ⋯ × A i r {\displaystyle A_{S}=A_{i_{1}}\times \cdots \times A_{i_{r}}} , p S = ( p i 1 , … , p i r ) {\displaystyle p_{S}=(p_{i_{1}},\ldots ,p_{i_{r}})} , q S = | A S | {\displaystyle q_{S}=|A_{S}|} , A = A 1 × ⋯ × A n {\displaystyle \mathbf {A} =A_{1}\times \cdots \times A_{n}} , and p = { p 1 , … , p n } {\displaystyle \mathbf {p} =\{p_{1},\ldots ,p_{n}\}} Let S = { S j } j = 1 M {\displaystyle S=\{S_{j}\}_{j=1}^{M}} where S j ⊂ { 1 , . . . , n } {\displaystyle S_{j}\subset \{1,...\,,n\}} . Suppose a function is defined as α i : A S i → R {\displaystyle \alpha _{i}:A_{S_{i}}\rightarrow R} , where R {\displaystyle R} is a commutative semiring. Also, p S i {\displaystyle p_{S_{i}}} are named the local domains and α i {\displaystyle \alpha _{i}} as the local kernels. Now the global kernel β : A → R {\displaystyle \beta :\mathbf {A} \rightarrow R} is defined as: β ( p 1 , . . . , p n ) = ∏ i = 1 M α ( p S i ) {\displaystyle \beta (p_{1},...\,,p_{n})=\prod _{i=1}^{M}\alpha (p_{S_{i}})} Definition of MPF problem: For one or more indices i = 1 , . . . , M {\displaystyle i=1,...\,,M} , compute a table of the values of S i {\displaystyle S_{i}} -marginalization of the global kernel β {\displaystyle \beta } , which is the function β i : A S i → R {\displaystyle \beta _{i}:A_{S_{i}}\rightarrow R} defined as β i ( p S i ) = ∑ p S i c ∈ A S i c β ( p ) {\displaystyle \beta _{i}(p_{S_{i}})\,=\displaystyle \sum \limits _{p_{S_{i}^{c}}\in A_{S_{i}^{c}}}\beta (p)} Here S i c {\displaystyle S_{i}^{c}} is the complement of S i {\displaystyle S_{i}} with respect to { 1 , . . . , n } {\displaystyle \mathbf {\{} 1,...\,,n\}} and the β i ( p S i ) {\displaystyle \beta _{i}(p_{S_{i}})} is called the i t h {\displaystyle i^{th}} objective function, or the objective function at S i {\displaystyle S_{i}} . It can observed that the computation of the i t h {\displaystyle i^{th}} objective function in the obvious way needs M q 1 q 2 q 3 ⋯ q n {\displaystyle Mq_{1}q_{2}q_{3}\cdots q_{n}} operations. This is because there are q 1 q 2 ⋯ q n {\displaystyle q_{1}q_{2}\cdots q_{n}} additions and ( M − 1 ) q 1 q 2 . . . q n {\displaystyle (M-1)q_{1}q_{2}...q_{n}} multiplications needed in the computation of the i th {\displaystyle i^{\text{th}}} objective function. The GDL algorithm which is explained in the next section can reduce this computational complexity. The following is an example of the MPF problem. Let p 1 , p 2 , p 3 , p 4 , {\displaystyle p_{1},\,p_{2},\,p_{3},\,p_{4},} and p 5 {\displaystyle p_{5}} be variables such that p 1 ∈ A 1 , p 2 ∈ A 2 , p 3 ∈ A 3 , p 4 ∈ A 4 , {\displaystyle p_{1}\in
System Service Descriptor Table
The System Service Descriptor Table (SSDT) is an internal dispatch table within Microsoft Windows. == Function == The SSDT maps syscalls to kernel function addresses. When a syscall is issued by a user space application, it contains the service index as parameter to indicate which syscall is called. The SSDT is then used to resolve the address of the corresponding function within ntoskrnl.exe. In modern Windows kernels, two SSDTs are used: One for generic routines (KeServiceDescriptorTable) and a second (KeServiceDescriptorTableShadow) for graphical routines. A parameter passed by the calling userspace application determines which SSDT shall be used. == Hooking == Modification of the SSDT allows to redirect syscalls to routines outside the kernel. These routines can be either used to hide the presence of software or to act as a backdoor to allow attackers permanent code execution with kernel privileges. For both reasons, hooking SSDT calls is often used as a technique in both Windows kernel mode rootkits and antivirus software. In 2010, many computer security products which relied on hooking SSDT calls were shown to be vulnerable to exploits using race conditions to attack the products' security checks.
Grammatical evolution
Grammatical evolution (GE) is a genetic programming (GP) technique (or approach) from evolutionary computation pioneered by Conor Ryan, JJ Collins and Michael O'Neill in 1998 at the BDS Group in the University of Limerick. As in any other GP approach, the objective is to find an executable program, program fragment, or function, which will achieve a good fitness value for a given objective function. In most published work on GP, a LISP-style tree-structured expression is directly manipulated, whereas GE applies genetic operators to an integer string, subsequently mapped to a program (or similar) through the use of a grammar, which is typically expressed in Backus–Naur form. One of the benefits of GE is that this mapping simplifies the application of search to different programming languages and other structures. == Problem addressed == In type-free, conventional Koza-style GP, the function set must meet the requirement of closure: all functions must be capable of accepting as their arguments the output of all other functions in the function set. Usually, this is implemented by dealing with a single data-type such as double-precision floating point. While modern Genetic Programming frameworks support typing, such type-systems have limitations that Grammatical Evolution does not suffer from. == GE's solution == GE offers a solution to the single-type limitation by evolving solutions according to a user-specified grammar (usually a grammar in Backus-Naur form). Therefore, the search space can be restricted, and domain knowledge of the problem can be incorporated. The inspiration for this approach comes from a desire to separate the "genotype" from the "phenotype": in GP, the objects the search algorithm operates on and what the fitness evaluation function interprets are one and the same. In contrast, GE's "genotypes" are ordered lists of integers which code for selecting rules from the provided context-free grammar. The phenotype, however, is the same as in Koza-style GP: a tree-like structure that is evaluated recursively. This model is more in line with how genetics work in nature, where there is a separation between an organism's genotype and the final expression of phenotype in proteins, etc. Separating genotype and phenotype allows a modular approach. In particular, the search portion of the GE paradigm needn't be carried out by any one particular algorithm or method. Observe that the objects GE performs search on are the same as those used in genetic algorithms. This means, in principle, that any existing genetic algorithm package, such as the popular GAlib, can be used to carry out the search, and a developer implementing a GE system need only worry about carrying out the mapping from list of integers to program tree. It is also in principle possible to perform the search using some other method, such as particle swarm optimization (see the remark below); the modular nature of GE creates many opportunities for hybrids as the problem of interest to be solved dictates. Brabazon and O'Neill have successfully applied GE to predicting corporate bankruptcy, forecasting stock indices, bond credit ratings, and other financial applications. GE has also been used with a classic predator-prey model to explore the impact of parameters such as predator efficiency, niche number, and random mutations on ecological stability. It is possible to structure a GE grammar that for a given function/terminal set is equivalent to genetic programming. == Criticism == Despite its successes, GE has been the subject of some criticism. One issue is that as a result of its mapping operation, GE's genetic operators do not achieve high locality which is a highly regarded property of genetic operators in evolutionary algorithms. == Variants == Although GE was originally described in terms of using an Evolutionary Algorithm, specifically, a Genetic Algorithm, other variants exist. For example, GE researchers have experimented with using particle swarm optimization to carry out the searching instead of genetic algorithms with results comparable to that of normal GE; this is referred to as a "grammatical swarm"; using only the basic PSO model it has been found that PSO is probably equally capable of carrying out the search process in GE as simple genetic algorithms are. (Although PSO is normally a floating-point search paradigm, it can be discretized, e.g., by simply rounding each vector to the nearest integer, for use with GE.) Yet another possible variation that has been experimented with in the literature is attempting to encode semantic information in the grammar in order to further bias the search process. Other work showed that, with biased grammars that leverage domain knowledge, even random search can be used to drive GE. == Related work == GE was originally a combination of the linear representation as used by the Genetic Algorithm for Developing Software (GADS) and Backus Naur Form grammars, which were originally used in tree-based GP by Wong and Leung in 1995 and Whigham in 1996. Other related work noted in the original GE paper was that of Frederic Gruau, who used a conceptually similar "embryonic" approach, as well as that of Keller and Banzhaf, which similarly used linear genomes. == Implementations == There are several implementations of GE. These include the following.
Independent component analysis
In signal processing, independent component analysis (ICA) is a computational method for separating a multivariate signal into additive subcomponents. This is done by assuming that at most one subcomponent is Gaussian and that the subcomponents are statistically independent from each other. ICA was invented by Jeanny Hérault and Christian Jutten in 1985. ICA is a special case of blind source separation. A common example application of ICA is the "cocktail party problem" of listening in on one person's speech in a noisy room. == Introduction == Independent component analysis attempts to decompose a multivariate signal into independent non-Gaussian signals. As an example, sound is usually a signal that is composed of the numerical addition, at each time t, of signals from several sources. The question then is whether it is possible to separate these contributing sources from the observed total signal. When the statistical independence assumption is correct, blind ICA separation of a mixed signal gives very good results. It is also used for signals that are not supposed to be generated by mixing for analysis purposes. A simple application of ICA is the "cocktail party problem", where the underlying speech signals are separated from a sample data consisting of people talking simultaneously in a room. Usually the problem is simplified by assuming no time delays or echoes. Note that a filtered and delayed signal is a copy of a dependent component, and thus the statistical independence assumption is not violated. Mixing weights for constructing the M {\textstyle M} observed signals from the N {\textstyle N} components can be placed in an M × N {\textstyle M\times N} matrix. An important thing to consider is that if N {\textstyle N} sources are present, at least N {\textstyle N} observations (e.g. microphones if the observed signal is audio) are needed to recover the original signals. When there are an equal number of observations and source signals, the mixing matrix is square ( M = N {\textstyle M=N} ). Other cases of underdetermined ( M < N {\textstyle M
Modes of variation
In statistics, modes of variation are a continuously indexed set of vectors or functions that are centered at a mean and are used to depict the variation in a population or sample. Typically, variation patterns in the data can be decomposed in descending order of eigenvalues with the directions represented by the corresponding eigenvectors or eigenfunctions. Modes of variation provide a visualization of this decomposition and an efficient description of variation around the mean. Both in principal component analysis (PCA) and in functional principal component analysis (FPCA), modes of variation play an important role in visualizing and describing the variation in the data contributed by each eigencomponent. In real-world applications, the eigencomponents and associated modes of variation aid to interpret complex data, especially in exploratory data analysis (EDA). == Formulation == Modes of variation are a natural extension of PCA and FPCA. === Modes of variation in PCA === If a random vector X = ( X 1 , X 2 , ⋯ , X p ) T {\displaystyle \mathbf {X} =(X_{1},X_{2},\cdots ,X_{p})^{T}} has the mean vector μ p {\displaystyle {\boldsymbol {\mu }}_{p}} , and the covariance matrix Σ p × p {\displaystyle \mathbf {\Sigma } _{p\times p}} with eigenvalues λ 1 ≥ λ 2 ≥ ⋯ ≥ λ p ≥ 0 {\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq \lambda _{p}\geq 0} and corresponding orthonormal eigenvectors e 1 , e 2 , ⋯ , e p {\displaystyle \mathbf {e} _{1},\mathbf {e} _{2},\cdots ,\mathbf {e} _{p}} , by eigendecomposition of a real symmetric matrix, the covariance matrix Σ {\displaystyle \mathbf {\Sigma } } can be decomposed as Σ = Q Λ Q T , {\displaystyle \mathbf {\Sigma } =\mathbf {Q} \mathbf {\Lambda } \mathbf {Q} ^{T},} where Q {\displaystyle \mathbf {Q} } is an orthogonal matrix whose columns are the eigenvectors of Σ {\displaystyle \mathbf {\Sigma } } , and Λ {\displaystyle \mathbf {\Lambda } } is a diagonal matrix whose entries are the eigenvalues of Σ {\displaystyle \mathbf {\Sigma } } . By the Karhunen–Loève expansion for random vectors, one can express the centered random vector in the eigenbasis X − μ = ∑ k = 1 p ξ k e k , {\displaystyle \mathbf {X} -{\boldsymbol {\mu }}=\sum _{k=1}^{p}\xi _{k}\mathbf {e} _{k},} where ξ k = e k T ( X − μ ) {\displaystyle \xi _{k}=\mathbf {e} _{k}^{T}(\mathbf {X} -{\boldsymbol {\mu }})} is the principal component associated with the k {\displaystyle k} -th eigenvector e k {\displaystyle \mathbf {e} _{k}} , with the properties E ( ξ k ) = 0 , Var ( ξ k ) = λ k , {\displaystyle \operatorname {E} (\xi _{k})=0,\operatorname {Var} (\xi _{k})=\lambda _{k},} and E ( ξ k ξ l ) = 0 for l ≠ k . {\displaystyle \operatorname {E} (\xi _{k}\xi _{l})=0\ {\text{for}}\ l\neq k.} Then the k {\displaystyle k} -th mode of variation of X {\displaystyle \mathbf {X} } is the set of vectors, indexed by α {\displaystyle \alpha } , m k , α = μ ± α λ k e k , α ∈ [ − A , A ] , {\displaystyle \mathbf {m} _{k,\alpha }={\boldsymbol {\mu }}\pm \alpha {\sqrt {\lambda _{k}}}\mathbf {e} _{k},\alpha \in [-A,A],} where A {\displaystyle A} is typically selected as 2 or 3 {\displaystyle 2\ {\text{or}}\ 3} . === Modes of variation in FPCA === For a square-integrable random function X ( t ) , t ∈ T ⊂ R p {\displaystyle X(t),t\in {\mathcal {T}}\subset R^{p}} , where typically p = 1 {\displaystyle p=1} and T {\displaystyle {\mathcal {T}}} is an interval, denote the mean function by μ ( t ) = E ( X ( t ) ) {\displaystyle \mu (t)=\operatorname {E} (X(t))} , and the covariance function by G ( s , t ) = Cov ( X ( s ) , X ( t ) ) = ∑ k = 1 ∞ λ k φ k ( s ) φ k ( t ) , {\displaystyle G(s,t)=\operatorname {Cov} (X(s),X(t))=\sum _{k=1}^{\infty }\lambda _{k}\varphi _{k}(s)\varphi _{k}(t),} where λ 1 ≥ λ 2 ≥ ⋯ ≥ 0 {\displaystyle \lambda _{1}\geq \lambda _{2}\geq \cdots \geq 0} are the eigenvalues and { φ 1 , φ 2 , ⋯ } {\displaystyle \{\varphi _{1},\varphi _{2},\cdots \}} are the orthonormal eigenfunctions of the linear Hilbert–Schmidt operator G : L 2 ( T ) → L 2 ( T ) , G ( f ) = ∫ T G ( s , t ) f ( s ) d s . {\displaystyle G:L^{2}({\mathcal {T}})\rightarrow L^{2}({\mathcal {T}}),\,G(f)=\int _{\mathcal {T}}G(s,t)f(s)ds.} By the Karhunen–Loève theorem, one can express the centered function in the eigenbasis, X ( t ) − μ ( t ) = ∑ k = 1 ∞ ξ k φ k ( t ) , {\displaystyle X(t)-\mu (t)=\sum _{k=1}^{\infty }\xi _{k}\varphi _{k}(t),} where ξ k = ∫ T ( X ( t ) − μ ( t ) ) φ k ( t ) d t {\displaystyle \xi _{k}=\int _{\mathcal {T}}(X(t)-\mu (t))\varphi _{k}(t)dt} is the k {\displaystyle k} -th principal component with the properties E ( ξ k ) = 0 , Var ( ξ k ) = λ k , {\displaystyle \operatorname {E} (\xi _{k})=0,\operatorname {Var} (\xi _{k})=\lambda _{k},} and E ( ξ k ξ l ) = 0 for l ≠ k . {\displaystyle \operatorname {E} (\xi _{k}\xi _{l})=0{\text{ for }}l\neq k.} Then the k {\displaystyle k} -th mode of variation of X ( t ) {\displaystyle X(t)} is the set of functions, indexed by α {\displaystyle \alpha } , m k , α ( t ) = μ ( t ) ± α λ k φ k ( t ) , t ∈ T , α ∈ [ − A , A ] {\displaystyle m_{k,\alpha }(t)=\mu (t)\pm \alpha {\sqrt {\lambda _{k}}}\varphi _{k}(t),\ t\in {\mathcal {T}},\ \alpha \in [-A,A]} that are viewed simultaneously over the range of α {\displaystyle \alpha } , usually for A = 2 or 3 {\displaystyle A=2\ {\text{or}}\ 3} . == Estimation == The formulation above is derived from properties of the population. Estimation is needed in real-world applications. The key idea is to estimate mean and covariance. === Modes of variation in PCA === Suppose the data x 1 , x 2 , ⋯ , x n {\displaystyle \mathbf {x} _{1},\mathbf {x} _{2},\cdots ,\mathbf {x} _{n}} represent n {\displaystyle n} independent drawings from some p {\displaystyle p} -dimensional population X {\displaystyle \mathbf {X} } with mean vector μ {\displaystyle {\boldsymbol {\mu }}} and covariance matrix Σ {\displaystyle \mathbf {\Sigma } } . These data yield the sample mean vector x ¯ {\displaystyle {\overline {\mathbf {x} }}} , and the sample covariance matrix S {\displaystyle \mathbf {S} } with eigenvalue-eigenvector pairs ( λ ^ 1 , e ^ 1 ) , ( λ ^ 2 , e ^ 2 ) , ⋯ , ( λ ^ p , e ^ p ) {\displaystyle ({\hat {\lambda }}_{1},{\hat {\mathbf {e} }}_{1}),({\hat {\lambda }}_{2},{\hat {\mathbf {e} }}_{2}),\cdots ,({\hat {\lambda }}_{p},{\hat {\mathbf {e} }}_{p})} . Then the k {\displaystyle k} -th mode of variation of X {\displaystyle \mathbf {X} } can be estimated by m ^ k , α = x ¯ ± α λ ^ k e ^ k , α ∈ [ − A , A ] . {\displaystyle {\hat {\mathbf {m} }}_{k,\alpha }={\overline {\mathbf {x} }}\pm \alpha {\sqrt {{\hat {\lambda }}_{k}}}{\hat {\mathbf {e} }}_{k},\alpha \in [-A,A].} === Modes of variation in FPCA === Consider n {\displaystyle n} realizations X 1 ( t ) , X 2 ( t ) , ⋯ , X n ( t ) {\displaystyle X_{1}(t),X_{2}(t),\cdots ,X_{n}(t)} of a square-integrable random function X ( t ) , t ∈ T {\displaystyle X(t),t\in {\mathcal {T}}} with the mean function μ ( t ) = E ( X ( t ) ) {\displaystyle \mu (t)=\operatorname {E} (X(t))} and the covariance function G ( s , t ) = Cov ( X ( s ) , X ( t ) ) {\displaystyle G(s,t)=\operatorname {Cov} (X(s),X(t))} . Functional principal component analysis provides methods for the estimation of μ ( t ) {\displaystyle \mu (t)} and G ( s , t ) {\displaystyle G(s,t)} in detail, often involving point wise estimate and interpolation. Substituting estimates for the unknown quantities, the k {\displaystyle k} -th mode of variation of X ( t ) {\displaystyle X(t)} can be estimated by m ^ k , α ( t ) = μ ^ ( t ) ± α λ ^ k φ ^ k ( t ) , t ∈ T , α ∈ [ − A , A ] . {\displaystyle {\hat {m}}_{k,\alpha }(t)={\hat {\mu }}(t)\pm \alpha {\sqrt {{\hat {\lambda }}_{k}}}{\hat {\varphi }}_{k}(t),t\in {\mathcal {T}},\alpha \in [-A,A].} == Applications == Modes of variation are useful to visualize and describe the variation patterns in the data sorted by the eigenvalues. In real-world applications, modes of variation associated with eigencomponents allow to interpret complex data, such as the evolution of function traits and other infinite-dimensional data. To illustrate how modes of variation work in practice, two examples are shown in the graphs to the right, which display the first two modes of variation. The solid curve represents the sample mean function. The dashed, dot-dashed, and dotted curves correspond to modes of variation with α = ± 1 , ± 2 , {\displaystyle \alpha =\pm 1,\pm 2,} and ± 3 {\displaystyle \pm 3} , respectively. The first graph displays the first two modes of variation of female mortality data from 41 countries in 2003. The object of interest is log hazard function between ages 0 and 100 years. The first mode of variation suggests that the variation of female mortality is smaller for ages around 0 or 100, and larger for ages around 25. An appropriate and intuitive interpretation is that mortality around 25 is driven by accidental death, while around 0 or 100, mortality is related to congenital disease or natural death. Compared to female mortality
Software engineering professionalism
Software engineering professionalism is a movement to make software engineering a profession, with aspects such as degree and certification programs, professional associations, professional ethics, and government licensing. The field is a licensed discipline in Texas in the United States (Texas Board of Professional Engineers, since 2013), Engineers Australia(Course Accreditation since 2001, not Licensing), and many provinces in Canada. == History == In 1993 the IEEE and ACM began a joint effort called JCESEP, which evolved into SWECC in 1998 to explore making software engineering into a profession. The ACM pulled out of SWECC in May 1999, objecting to its support for the Texas professionalization efforts, of having state licenses for software engineers. ACM determined that the state of knowledge and practice in software engineering was too immature to warrant licensing, and that licensing would give false assurances of competence even if the body of knowledge were mature. The IEEE continued to support making software engineering a branch of traditional engineering. In Canada the Canadian Information Processing Society established the Information Systems Professional certification process. Also, by the late 1990s (1999 in British Columbia) the discipline of software engineering as a professional engineering discipline was officially created. This has caused some disputes between the provincial engineering associations and companies who call their developers software engineers, even though these developers have not been licensed by any engineering association. In 1999, the Panel of Software Engineering was formed as part of the settlement between Engineering Canada and the Memorial University of Newfoundland over the school's use of the term "software engineering" in the name of a computer science program. Concerns were raised over the inappropriate use of the name "software engineering" to describe non-engineering programs could lead to student and public confusion, and ultimately threaten public safety. The Panel issued recommendations to create a Software Engineering Accreditation Board, but the task force created to carry out the recommendations was unable to get the various stakeholders to agree to concrete proposals, resulting in separate accreditation boards. == Ethics == Software engineering ethics is a large field. In some ways it began as an unrealistic attempt to define bugs as unethical. More recently it has been defined as the application of both computer science and engineering philosophy, principles, and practices to the design and development of software systems. Due to this engineering focus and the increased use of software in mission critical and human critical systems, where failure can result in large losses of capital but more importantly lives such as the Therac-25 system, many ethical codes have been developed by a number of societies, associations and organizations. These entities, such as the ACM, IEEE, EGBC and Institute for Certification of Computing Professionals (ICCP) have formal codes of ethics. Adherence to the code of ethics is required as a condition of membership or certification. According to the ICCP, violation of the code can result in revocation of the certificate. Also, all engineering societies require conformance to their ethical codes; violation of the code results in the revocation of the license to practice engineering in the society's jurisdiction. These codes of ethics usually have much in common. They typically relate the need to act consistently with the client's interest, employer's interest, and most importantly the public's interest. They also outline the need to act with professionalism and to promote an ethical approach to the profession. A Software Engineering Code of Ethics has been approved by the ACM and the IEEE-CS as the standard for teaching and practicing software engineering. === Examples of codes of conduct === The following are examples of codes of conduct for Professional Engineers. These 2 have been chosen because both jurisdictions have a designation for Professional Software Engineers. Engineers and Geoscientists of British Columbia (EGBC): All members in the association's code of Ethics must ensure that the government, the public can rely on BC's professional engineers and Geoscientists to act at all times with fairness, courtesy and good faith to their employers, employee and customers, and to uphold the truth, honesty and trustworthiness, and to safe guard human life and the environment. This is just one of the many ways in which BC's Professional Engineers and Professional Geoscientists maintain their competitive edge in today's global marketplace. Association of Professional Engineers and Geoscientists of Alberta (APEGA): Different with British Columbia, the Alberta Government granted self governance to engineers, Geoscientists and geophysicists. All members in the APEGA have to accept legal and ethical responsibility for the work and to hold the interest of the public and society. The APEGA is a standards guideline of professional practice to uphold the protection of public interest for engineering, Geoscientists and geophysics in Alberta. === Opinions on ethics === Bill Joy argued that "better software" can only enable its privileged end users, make reality more power-pointy as opposed to more humane, and ultimately run away with itself so that "the future doesn't need us." He openly questioned the goals of software engineering in this respect, asking why it isn't trying to be more ethical rather than more efficient. In his book Code and Other Laws of Cyberspace, Lawrence Lessig argues that computer code can regulate conduct in much the same way as the legal code. Lessig and Joy urge people to think about the consequences of the software being developed, not only in a functional way, but also in how it affects the public and society as a whole. Overall, due to the youth of software engineering, many of the ethical codes and values have been borrowed from other fields, such as mechanical and civil engineering. However, there are many ethical questions that even these, much older, disciplines have not encountered. Questions about the ethical impact of internet applications, which have a global reach, have never been encountered until recently and other ethical questions are still to be encountered. This means the ethical codes for software engineering are a work in progress, that will change and update as more questions arise. == Independent licensing and certification exams == Since 2002, the IEEE Computer Society offered the Certified Software Development Professional (CSDP) certification exam (in 2015 this was replaced by several similar certifications). A group of experts from industry and academia developed the exam and maintained it. Donald Bagert, and at a later period Stephen Tockey headed the certification committee. Contents of the exam centered around the SWEBOK (Software Engineering Body of Knowledge) guide, with an additional emphasis on Professional Practices and Software Engineering Economics knowledge areas (KAs). The motivation was to produce a structure at an international level for software engineering's knowledge areas. == Criticism of licensing == Professional licensing has been criticized for many reasons. The field of software engineering is too immature Licensing would give false assurances of competence even if the body of knowledge were mature Software engineers would have to study years of calculus, physics, and chemistry to pass the exams, which is irrelevant to most software practitioners. Many (most?) computer science majors don't earn degrees in engineering schools, so they are probably unqualified to pass engineering exams. == Licensing by country == === United States === The Bureau of Labor Statistics (BLS) classifies computer software engineers as a subcategory of "computer specialists", along with occupations such as computer scientist, Programmer, Database administrator and Network administrator. The BLS classifies all other engineering disciplines, including computer hardware engineers, as engineers. Many states prohibit unlicensed persons from calling themselves an Engineer, or from indicating branches or specialties not covered licensing acts. In many states, the title Engineer is reserved for individuals with a Professional Engineering license indicating that they have shown minimum level of competency through accredited engineering education, qualified engineering experience, and engineering board's examinations. In April 2013 the National Council of Examiners for Engineering and Surveying (NCEES) began offering a Professional Engineer (PE) exam for Software Engineering. The exam was developed in association with the IEEE Computer Society. NCEES ended the exam in April 2019 due to lack of participation. The American National Society of Professional Engineers provides a model law and lobbies legislatures to adopt occ
Farthest-first traversal
In computational geometry, the farthest-first traversal of a compact metric space is a sequence of points in the space, where the first point is selected arbitrarily and each successive point is as far as possible from the set of previously-selected points. The same concept can also be applied to a finite set of geometric points, by restricting the selected points to belong to the set or equivalently by considering the finite metric space generated by these points. For a finite metric space or finite set of geometric points, the resulting sequence forms a permutation of the points, also known as the greedy permutation. Every prefix of a farthest-first traversal provides a set of points that is widely spaced and close to all remaining points. More precisely, no other set of equally many points can be spaced more than twice as widely, and no other set of equally many points can be less than half as far to its farthest remaining point. In part because of these properties, farthest-point traversals have many applications, including the approximation of the traveling salesman problem and the metric k-center problem. They may be constructed in polynomial time, or (for low-dimensional Euclidean spaces) approximated in near-linear time. == Definition and properties == A farthest-first traversal is a sequence of points in a compact metric space, with each point appearing at most once. If the space is finite, each point appears exactly once, and the traversal is a permutation of all of the points in the space. The first point of the sequence may be any point in the space. Each point p after the first must have the maximum possible distance to the set of points earlier than p in the sequence, where the distance from a point to a set is defined as the minimum of the pairwise distances to points in the set. A given space may have many different farthest-first traversals, depending both on the choice of the first point in the sequence (which may be any point in the space) and on ties for the maximum distance among later choices. Farthest-point traversals may be characterized by the following properties. Fix a number k, and consider the prefix formed by the first k points of the farthest-first traversal of any metric space. Let r be the distance between the final point of the prefix and the other points in the prefix. Then this subset has the following two properties: All pairs of the selected points are at distance at least r from each other, and All points of the metric space are at distance at most r from the subset. Conversely any sequence having these properties, for all choices of k, must be a farthest-first traversal. These are the two defining properties of a Delone set, so each prefix of the farthest-first traversal forms a Delone set. == Applications == Rosenkrantz, Stearns & Lewis (1977) used the farthest-first traversal to define the farthest-insertion heuristic for the travelling salesman problem. This heuristic finds approximate solutions to the travelling salesman problem by building up a tour on a subset of points, adding one point at a time to the tour in the ordering given by a farthest-first traversal. To add each point to the tour, one edge of the previous tour is broken and replaced by a pair of edges through the added point, in the cheapest possible way. Although Rosenkrantz et al. prove only a logarithmic approximation ratio for this method, they show that in practice it often works better than other insertion methods with better provable approximation ratios. Later, the same sequence of points was popularized by Gonzalez (1985), who used it as part of greedy approximation algorithms for two problems in clustering, in which the goal is to partition a set of points into k clusters. One of the two problems that Gonzalez solve in this way seeks to minimize the maximum diameter of a cluster, while the other, known as the metric k-center problem, seeks to minimize the maximum radius, the distance from a chosen central point of a cluster to the farthest point from it in the same cluster. For instance, the k-center problem can be used to model the placement of fire stations within a city, in order to ensure that every address within the city can be reached quickly by a fire truck. For both clustering problems, Gonzalez chooses a set of k cluster centers by selecting the first k points of a farthest-first traversal, and then creates clusters by assigning each input point to the nearest cluster center. If r is the distance from the set of k selected centers to the next point at position k + 1 in the traversal, then with this clustering every point is within distance r of its center and every cluster has diameter at most 2r. However, the subset of k centers together with the next point are all at distance at least r from each other, and any k-clustering would put some two of these points into a single cluster, with one of them at distance at least r/2 from its center and with diameter at least r. Thus, Gonzalez's heuristic gives an approximation ratio of 2 for both clustering problems. Gonzalez's heuristic was independently rediscovered for the metric k-center problem by Dyer & Frieze (1985), who applied it more generally to weighted k-center problems. Another paper on the k-center problem from the same time, Hochbaum & Shmoys (1985), achieves the same approximation ratio of 2, but its techniques are different. Nevertheless, Gonzalez's heuristic, and the name "farthest-first traversal", are often incorrectly attributed to Hochbaum and Shmoys. For both the min-max diameter clustering problem and the metric k-center problem, these approximations are optimal: the existence of a polynomial-time heuristic with any constant approximation ratio less than 2 would imply that P = NP. As well as for clustering, the farthest-first traversal can also be used in another type of facility location problem, the max-min facility dispersion problem, in which the goal is to choose the locations of k different facilities so that they are as far apart from each other as possible. More precisely, the goal in this problem is to choose k points from a given metric space or a given set of candidate points, in such a way as to maximize the minimum pairwise distance between the selected points. Again, this can be approximated by choosing the first k points of a farthest-first traversal. If r denotes the distance of the kth point from all previous points, then every point of the metric space or the candidate set is within distance r of the first k − 1 points. By the pigeonhole principle, some two points of the optimal solution (whatever it is) must both be within distance r of the same point among these first k − 1 chosen points, and (by the triangle inequality) within distance 2r of each other. Therefore, the heuristic solution given by the farthest-first traversal is within a factor of two of optimal. Other applications of the farthest-first traversal include color quantization (clustering the colors in an image to a smaller set of representative colors), progressive scanning of images (choosing an order to display the pixels of an image so that prefixes of the ordering produce good lower-resolution versions of the whole image rather than filling in the image from top to bottom), point selection in the probabilistic roadmap method for motion planning, simplification of point clouds, generating masks for halftone images, hierarchical clustering, finding the similarities between polygon meshes of similar surfaces, choosing diverse and high-value observation targets for underwater robot exploration, fault detection in sensor networks, modeling phylogenetic diversity, matching vehicles in a heterogenous fleet to customer delivery requests, uniform distribution of geodetic observatories on the Earth's surface or of other types of sensor network, generation of virtual point lights in the instant radiosity computer graphics rendering method, and geometric range searching data structures. == Algorithms == === Greedy exact algorithm === The farthest-first traversal of a finite point set may be computed by a greedy algorithm that maintains the distance of each point from the previously selected points, performing the following steps: Initialize the sequence of selected points to the empty sequence, and the distances of each point to the selected points to infinity. While not all points have been selected, repeat the following steps: Scan the list of not-yet-selected points to find a point p that has the maximum distance from the selected points. Remove p from the not-yet-selected points and add it to the end of the sequence of selected points. For each remaining not-yet-selected point q, replace the distance stored for q by the minimum of its old value and the distance from p to q. For a set of n points, this algorithm takes O(n2) steps and O(n2) distance computations. === Approximations === A faster approximation algorithm, given by Har-Peled & Mendel (2006), applie