In machine learning, semantic analysis of a text corpus is the task of building structures that approximate concepts from a large set of documents. It generally does not involve prior semantic understanding of the documents. Semantic analysis strategies include: Metalanguages based on first-order logic, which can analyze the speech of humans. Understanding the semantics of a text is symbol grounding: if language is grounded, it is equal to recognizing a machine-readable meaning. For the restricted domain of spatial analysis, a computer-based language understanding system was demonstrated. Latent semantic analysis (LSA), a class of techniques where documents are represented as vectors in a term space. A prominent example is probabilistic latent semantic analysis (PLSA). Latent Dirichlet allocation, which involves attributing document terms to topics. n-grams and hidden Markov models, which work by representing the term stream as a Markov chain, in which each term is derived from preceding terms. == Stochastic semantic analysis ==
Commitment ordering
Commitment ordering (CO) is a class of interoperable serializability techniques in concurrency control of databases, transaction processing, and related applications. It allows optimistic (non-blocking) implementations. With the proliferation of multi-core processors, CO has also been increasingly utilized in concurrent programming, transactional memory, and software transactional memory (STM) to achieve serializability optimistically. CO is also the name of the resulting transaction schedule (history) property, defined in 1988 with the name dynamic atomicity. In a CO compliant schedule, the chronological order of commitment events of transactions is compatible with the precedence order of the respective transactions. CO is a broad special case of conflict serializability and effective means (reliable, high-performance, distributed, and scalable) to achieve global serializability (modular serializability) across any collection of database systems that possibly use different concurrency control mechanisms (CO also makes each system serializability compliant, if not already). Each not-CO-compliant database system is augmented with a CO component (the commitment order coordinator—COCO) which orders the commitment events for CO compliance, with neither data-access nor any other transaction operation interference. As such, CO provides a low overhead, general solution for global serializability (and distributed serializability), instrumental for global concurrency control (and distributed concurrency control) of multi-database systems and other transactional objects, possibly highly distributed (e.g., within cloud computing, grid computing, and networks of smartphones). An atomic commitment protocol (ACP; of any type) is a fundamental part of the solution, utilized to break global cycles in the conflict (precedence, serializability) graph. CO is the most general property (a necessary condition) that guarantees global serializability, if the database systems involved do not share concurrency control information beyond atomic commitment protocol (unmodified) messages and have no knowledge of whether transactions are global or local (the database systems are autonomous). Thus CO (with its variants) is the only general technique that does not require the typically costly distribution of local concurrency control information (e.g., local precedence relations, locks, timestamps, or tickets). It generalizes the popular strong strict two-phase locking (SS2PL) property, which in conjunction with the two-phase commit protocol (2PC), is the de facto standard to achieve global serializability across (SS2PL based) database systems. As a result, CO compliant database systems (with any different concurrency control types) can transparently join such SS2PL based solutions for global serializability. In addition, locking based global deadlocks are resolved automatically in a CO based multi-database environment, a vital side-benefit (including the special case of a completely SS2PL based environment; a previously unnoticed fact for SS2PL). Furthermore, strict commitment ordering (SCO; Raz 1991c), the intersection of Strictness and CO, provides better performance (shorter average transaction completion time and resulting in better transaction throughput) than SS2PL whenever read-write conflicts are present (identical blocking behavior for write-read and write-write conflicts; comparable locking overhead). The advantage of SCO is especially during lock contention. Strictness allows both SS2PL and SCO to use the same effective database recovery mechanisms. Two major generalizing variants of CO exist, extended CO (ECO; Raz 1993a) and multi-version CO (MVCO; Raz 1993b). They also provide global serializability without local concurrency control information distribution, can be combined with any relevant concurrency control, and allow optimistic (non-blocking) implementations. Both use additional information for relaxing CO constraints and achieving better concurrency and performance. Vote ordering (VO or Generalized CO (GCO); Raz 2009) is a container schedule set (property) and technique for CO and all its variants. Local VO is necessary for guaranteeing global serializability if the atomic commitment protocol (ACP) participants do not share concurrency control information (have the generalized autonomy property). CO and its variants inter-operate transparently, guaranteeing global serializability and automatic global deadlock resolution together in a mixed, heterogeneous environment with different variants. == Overview == The Commitment ordering (CO; Raz 1990, 1992, 1994, 2009) schedule property has been referred to also as Dynamic atomicity (since 1988), commit ordering, commit order serializability, and strong recoverability (since 1991). The latter is a misleading name since CO is incomparable with recoverability, and the term "strong" implies a special case. This means that a substantial recoverability property does not necessarily have the CO property and vice versa. In 2009 CO has been characterized as a major concurrency control method, together with the previously known (since the 1980s) three major methods: Locking, Time-stamp ordering, and Serialization graph testing, and as an enabler for the interoperability of systems using different concurrency control mechanisms. In a federated database system or any other more loosely defined multidatabase system, which are typically distributed in a communication network, transactions span multiple and possibly Distributed databases. Enforcing global serializability in such system is problematic. Even if every local schedule of a single database is still serializable, the global schedule of a whole system is not necessarily serializable. The massive communication exchanges of conflict information needed between databases to reach conflict serializability would lead to unacceptable performance, primarily due to computer and communication latency. The problem of achieving global serializability effectively had been characterized as open until the public disclosure of CO in 1991 by its inventor Yoav Raz (Raz 1991a; see also Global serializability). Enforcing CO is an effective way to enforce conflict serializability globally in a distributed system since enforcing CO locally in each database (or other transactional objects) also enforces it globally. Each database may use any, possibly different, type of concurrency control mechanism. With a local mechanism that already provides conflict serializability, enforcing CO locally does not cause any other aborts, since enforcing CO locally does not affect the data access scheduling strategy of the mechanism (this scheduling determines the serializability related aborts; such a mechanism typically does not consider the commitment events or their order). The CO solution requires no communication overhead since it uses (unmodified) atomic commitment protocol messages only, already needed by each distributed transaction to reach atomicity. An atomic commitment protocol plays a central role in the distributed CO algorithm, which enforces CO globally by breaking global cycles (cycles that span two or more databases) in the global conflict graph. CO, its special cases, and its generalizations are interoperable and achieve global serializability while transparently being utilized together in a single heterogeneous distributed environment comprising objects with possibly different concurrency control mechanisms. As such, Commitment ordering, including its special cases, and together with its generalizations (see CO variants below), provides a general, high performance, fully distributed solution (no central processing component or central data structure are needed) for guaranteeing global serializability in heterogeneous environments of multidatabase systems and other multiple transactional objects (objects with states accessed and modified only by transactions; e.g., in the framework of transactional processes, and within Cloud computing and Grid computing). The CO solution scales up with network size and the number of databases without any negative impact on performance (assuming the statistics of a single distributed transaction, e.g., the average number of databases involved with a single transaction, are unchanged). With the proliferation of Multi-core processors, Optimistic CO (OCO) has also been increasingly utilized to achieve serializability in software transactional memory, and numerous STM articles and patents utilizing "commit order" have already been published (e.g., Zhang et al. 2006). == The commitment ordering solution for global serializability == === General characterization of CO === Commitment ordering (CO) is a special case of conflict serializability. CO can be enforced with non-blocking mechanisms (each transaction can complete its task without having its data-access blocked, which allows optimistic concurrency control; however, commitment could be blo
Gaussian process emulator
In statistics, Gaussian process emulator is one name for a general type of statistical model that has been used in contexts where the problem is to make maximum use of the outputs of a complicated (often non-random) computer-based simulation model. Each run of the simulation model is computationally expensive and each run is based on many different controlling inputs. The variation of the outputs of the simulation model is expected to vary reasonably smoothly with the inputs, but in an unknown way. The overall analysis involves two models: the simulation model, or "simulator", and the statistical model, or "emulator", which notionally emulates the unknown outputs from the simulator. The Gaussian process emulator model treats the problem from the viewpoint of Bayesian statistics. In this approach, even though the output of the simulation model is fixed for any given set of inputs, the actual outputs are unknown unless the computer model is run and hence can be made the subject of a Bayesian analysis. The main element of the Gaussian process emulator model is that it models the outputs as a Gaussian process on a space that is defined by the model inputs. The model includes a description of the correlation or covariance of the outputs, which enables the model to encompass the idea that differences in the output will be small if there are only small differences in the inputs.
Time-aware long short-term memory
Time-aware LSTM (T-LSTM) is a long short-term memory (LSTM) unit capable of handling irregular time intervals in longitudinal patient records. T-LSTM was developed by researchers from Michigan State University, IBM Research, and Cornell University and was first presented in the Knowledge Discovery and Data Mining (KDD) conference. Experiments using real and synthetic data proved that T-LSTM auto-encoder outperformed widely used frameworks including LSTM and MF1-LSTM auto-encoders.
ARKA descriptors in QSAR
In computational chemistry and cheminformatics, ARKA descriptors in QSAR are a class of molecular descriptors used in quantitative structure–activity relationship (QSAR) modeling (or related approaches such as QSPR and QSTR), a computational method for predicting the biological activity or toxicity of chemical compounds based on their molecular structure. Molecular descriptors are numerical values that summarize information about a molecule's structure, topology, geometry, or physicochemical properties in a form suitable for machine learning or statistical modeling. ARKA (Arithmetic Residuals in K-Groups Analysis) descriptors differ from traditional descriptors by encoding atomic-level information through recursive autoregression techniques, which aim to capture subtle structural patterns and improve predictive accuracy. They are designed to be both interpretable and well-suited to modeling nonlinear relationships in QSAR studies. == Comparisons == While QSAR is essentially a similarity-based approach, the occurrence of activity/property cliffs may greatly reduce the predictive accuracy of the developed models. The novel Arithmetic Residuals in K-groups Analysis (ARKA) approach is a supervised dimensionality reduction technique developed by the DTC Laboratory, Jadavpur University that can easily identify activity cliffs in a data set. Activity cliffs are similar in their structures but differ considerably in their activity. The basic idea of the ARKA descriptors is to group the conventional QSAR descriptors based on a predefined criterion and then assign weightage to each descriptor in each group. ARKA descriptors have also been used to develop classification-based and regression-based QSAR models with acceptable quality statistics. The ARKA descriptors have been used for the identification of activity cliffs in QSAR studies and/or model development by multiple researchers. A tutorial presentation on the ARKA descriptors is available. Recently a multi-class ARKA framework has been proposed for improved q-RASAR model generation.
Richardson–Lucy deconvolution
The Richardson–Lucy algorithm, also known as Lucy–Richardson deconvolution, is an iterative procedure for recovering an underlying image that has been blurred by a known point spread function. It was named after William Richardson and Leon B. Lucy, who described it independently. == Description == When an image is produced using an optical system and detected using photographic film, a charge-coupled device or a CMOS sensor, for example, it is inevitably blurred, with an ideal point source not appearing as a point but being spread out into what is known as the point spread function. Extended sources can be decomposed into the sum of many individual point sources, thus the observed image can be represented in terms of a transition matrix p operating on an underlying image: d i = ∑ j p i , j u j , {\displaystyle d_{i}=\sum _{j}p_{i,j}u_{j},} where u j {\displaystyle u_{j}} is the intensity of the underlying image at pixel j {\displaystyle j} , and d i {\displaystyle d_{i}} is the detected intensity at pixel i {\displaystyle i} . In general, a matrix whose elements are p i , j {\displaystyle p_{i,j}} describes the portion of light from source pixel j that is detected in pixel i. In most good optical systems (or in general, linear systems that are described as shift-invariant) the transfer function p can be expressed simply in terms of the spatial offset between the source pixel j and the observation pixel i: p i , j = P ( i − j ) , {\displaystyle p_{i,j}=P(i-j),} where P ( Δ i ) {\displaystyle P(\Delta i)} is called a point spread function. In that case the above equation becomes a convolution. This has been written for one spatial dimension, but most imaging systems are two-dimensional, with the source, detected image, and point spread function all having two indices. So a two-dimensional detected image is a convolution of the underlying image with a two-dimensional point spread function P ( Δ x , Δ y ) {\displaystyle P(\Delta x,\Delta y)} plus added detection noise. In order to estimate u j {\displaystyle u_{j}} given the observed d i {\displaystyle d_{i}} and a known P ( Δ i x , Δ j y ) {\displaystyle P(\Delta i_{x},\Delta j_{y})} , the following iterative procedure is employed in which the estimate of u j {\displaystyle u_{j}} (called u ^ j ( t ) {\displaystyle {\hat {u}}_{j}^{(t)}} ) for iteration number t is updated as follows: u ^ j ( t + 1 ) = u ^ j ( t ) ∑ i d i c i p i j , {\displaystyle {\hat {u}}_{j}^{(t+1)}={\hat {u}}_{j}^{(t)}\sum _{i}{\frac {d_{i}}{c_{i}}}p_{ij},} where c i = ∑ j p i j u ^ j ( t ) , {\displaystyle c_{i}=\sum _{j}p_{ij}{\hat {u}}_{j}^{(t)},} and ∑ j p i j = 1 {\displaystyle \sum _{j}p_{ij}=1} is assumed. It has been shown empirically that if this iteration converges, it converges to the maximum likelihood solution for u j {\displaystyle u_{j}} . Writing this more generally for two (or more) dimensions in terms of convolution with a point spread function P: u ^ ( t + 1 ) = u ^ ( t ) ⋅ ( d u ^ ( t ) ⊗ P ⊗ P ∗ ) , {\displaystyle {\hat {u}}^{(t+1)}={\hat {u}}^{(t)}\cdot \left({\frac {d}{{\hat {u}}^{(t)}\otimes P}}\otimes P^{}\right),} where the division and multiplication are element-wise, ⊗ {\displaystyle \otimes } indicates a 2D convolution, and P ∗ {\displaystyle P^{}} is the mirrored point spread function, or the inverse Fourier transform of the Hermitian transpose of the optical transfer function. In problems where the point spread function p i j {\displaystyle p_{ij}} is not known a priori, a modification of the Richardson–Lucy algorithm has been proposed, in order to accomplish blind deconvolution. == Derivation == In the context of fluorescence microscopy, the probability of measuring a set of number of photons (or digitalization counts proportional to detected light) m = [ m 0 , … , m K ] {\displaystyle \mathbf {m} =[m_{0},\dots ,m_{K}]} for expected values E = [ E 0 , … , E K ] {\displaystyle \mathbf {E} =[E_{0},\dots ,E_{K}]} for a detector with K + 1 {\displaystyle K+1} pixels is given by P ( m ∣ E ) = ∏ i K Poisson ( E i ) = ∏ i K E i m i e − E i m i ! . {\displaystyle P(\mathbf {m} \mid \mathbf {E} )=\prod _{i}^{K}\operatorname {Poisson} (E_{i})=\prod _{i}^{K}{\frac {E_{i}^{m_{i}}e^{-E_{i}}}{m_{i}!}}.} Since in the context of maximum-likelihood estimation the aim is to locate the maximum of the likelihood function without concern for its absolute value, it is convenient to work with ln ( P ) {\displaystyle \ln(P)} : ln P ( m ∣ E ) = ∑ i K [ ( m i ln E i − E i ) − ln ( m i ! ) ] . {\displaystyle \ln P(\mathbf {m} \mid \mathbf {E} )=\sum _{i}^{K}[(m_{i}\ln E_{i}-E_{i})-\ln(m_{i}!)].} Moreover, since ln ( m i ! ) {\displaystyle \ln(m_{i}!)} is a constant, it does not give any additional information regarding the position of the maximum, so consider α ( m ∣ E ) = ∑ i K [ m i ln E i − E i ] , {\displaystyle \alpha (\mathbf {m} \mid \mathbf {E} )=\sum _{i}^{K}[m_{i}\ln E_{i}-E_{i}],} where α {\displaystyle \alpha } is something that shares the same maximum position as P ( m ∣ E ) {\displaystyle P(\mathbf {m} \mid \mathbf {E} )} . Now consider that E {\displaystyle \mathbf {E} } comes from a ground truth x {\displaystyle \mathbf {x} } and a measurement H {\displaystyle \mathbf {H} } which is assumed to be linear. Then E = H x , {\displaystyle \mathbf {E} =\mathbf {H} \mathbf {x} ,} where a matrix multiplication is implied. This can also be written in the form E m = ∑ n K H m n x n , {\displaystyle E_{m}=\sum _{n}^{K}H_{mn}x_{n},} where it can be seen how H {\displaystyle H} mixes or blurs the ground truth. It can also be shown that the derivative of an element of E {\displaystyle \mathbf {E} } , ( E i ) {\displaystyle (E_{i})} with respect to some other element of x j {\displaystyle x_{j}} can be written as It is easy to see this by writing a matrix H {\displaystyle \mathbf {H} } of, say, 5 × 5 and two arrays E {\displaystyle \mathbf {E} } and x {\displaystyle \mathbf {x} } of 5 elements and check it. This last equation can be interpreted as how much one element of x {\displaystyle \mathbf {x} } , say element i {\displaystyle i} , influences the other elements j ≠ i {\displaystyle j\neq i} (and of course the case i = j {\displaystyle i=j} is also taken into account). For example, in a typical case an element of the ground truth x {\displaystyle \mathbf {x} } will influence nearby elements in E {\displaystyle \mathbf {E} } but not the very distant ones (a value of 0 {\displaystyle 0} is expected on those matrix elements). Now, the key and arbitrary step: x {\displaystyle \mathbf {x} } is not known but may be estimated by x ^ {\displaystyle {\hat {\mathbf {x} }}} . Let's call x ^ old {\displaystyle {\hat {\mathbf {x} }}_{\text{old}}} and x ^ new {\displaystyle {\hat {\mathbf {x} }}_{\text{new}}} the estimated ground truths while using the RL algorithm, where the hat symbol is used to distinguish ground truth from estimator of the ground truth where ∂ ∂ x {\displaystyle {\frac {\partial }{\partial \mathbf {x} }}} stands for a K {\displaystyle K} -dimensional gradient. Performing the partial derivative of α ( m ∣ E ( x ) ) {\displaystyle \alpha (\mathbf {m} \mid \mathbf {E} (\mathbf {x} ))} yields the following expression: ∂ α ( m ∣ E ( x ) ) ∂ x j = ∂ ∂ x j ∑ i K [ m i ln E i − E i ] = ∑ i K [ m i E i ∂ ∂ x j E i − ∂ ∂ x j E i ] = ∑ i K ∂ E i ∂ x j [ m i E i − 1 ] . {\displaystyle {\frac {\partial \alpha (\mathbf {m} \mid \mathbf {E} (\mathbf {x} ))}{\partial x_{j}}}={\frac {\partial }{\partial x_{j}}}\sum _{i}^{K}[m_{i}\ln E_{i}-E_{i}]=\sum _{i}^{K}\left[{\frac {m_{i}}{E_{i}}}{\frac {\partial }{\partial x_{j}}}E_{i}-{\frac {\partial }{\partial x_{j}}}E_{i}\right]=\sum _{i}^{K}{\frac {\partial E_{i}}{\partial x_{j}}}\left[{\frac {m_{i}}{E_{i}}}-1\right].} By substituting (1), it follows that ∂ α ( m ∣ E ( x ) ) ∂ x j = ∑ i K H i j [ m i E i − 1 ] . {\displaystyle {\frac {\partial \alpha (\mathbf {m} \mid \mathbf {E} (\mathbf {x} ))}{\partial x_{j}}}=\sum _{i}^{K}H_{ij}\left[{\frac {m_{i}}{E_{i}}}-1\right].} Note that H j i T = H i j {\displaystyle H_{ji}^{T}=H_{ij}} by the definition of a matrix transpose. And hence Since this equation is true for all j {\displaystyle j} spanning all the elements from 1 {\displaystyle 1} to K {\displaystyle K} , these K {\displaystyle K} equations may be compactly rewritten as a single vectorial equation ∂ α ( m ∣ E ( x ) ) ∂ x = H T [ m E − 1 ] , {\displaystyle {\frac {\partial \alpha (\mathbf {m} \mid \mathbf {E} (\mathbf {x} ))}{\partial \mathbf {x} }}=\mathbf {H} ^{T}\left[{\frac {\mathbf {m} }{\mathbf {E} }}-\mathbf {1} \right],} where H T {\displaystyle \mathbf {H} ^{T}} is a matrix, and m {\displaystyle \mathbf {m} } , E {\displaystyle \mathbf {E} } and 1 {\displaystyle \mathbf {1} } are vectors. Now, as a seemingly arbitrary but key step, let where 1 {\displaystyle \mathbf {1} } is a vector of ones of size K {\displaystyle K} (same as m {\displaystyle \mathbf {m} } , E {\displaystyle \mathbf {E} } and x {\displaystyle \mathbf {x} } ), and the d
Variable kernel density estimation
In statistics, adaptive or "variable-bandwidth" kernel density estimation is a form of kernel density estimation in which the size of the kernels used in the estimate are varied depending upon either the location of the samples or the location of the test point. It is a particularly effective technique when the sample space is multi-dimensional. == Rationale == Given a set of samples, { x → i } {\displaystyle \lbrace {\vec {x}}_{i}\rbrace } , we wish to estimate the density, P ( x → ) {\displaystyle P({\vec {x}})} , at a test point, x → {\displaystyle {\vec {x}}} : P ( x → ) ≈ W n h D {\displaystyle P({\vec {x}})\approx {\frac {W}{nh^{D}}}} W = ∑ i = 1 n w i {\displaystyle W=\sum _{i=1}^{n}w_{i}} w i = K ( x → − x → i h ) {\displaystyle w_{i}=K\left({\frac {{\vec {x}}-{\vec {x}}_{i}}{h}}\right)} where n is the number of samples, K is the "kernel", h is its width and D is the number of dimensions in x → {\displaystyle {\vec {x}}} . The kernel can be thought of as a simple, linear filter. Using a fixed filter width may mean that in regions of low density, all samples will fall in the tails of the filter with very low weighting, while regions of high density will find an excessive number of samples in the central region with weighting close to unity. To fix this problem, we vary the width of the kernel in different regions of the sample space. There are two methods of doing this: balloon and pointwise estimation. In a balloon estimator, the kernel width is varied depending on the location of the test point. In a pointwise estimator, the kernel width is varied depending on the location of the sample. For multivariate estimators, the parameter, h, can be generalized to vary not just the size, but also the shape of the kernel. This more complicated approach will not be covered here. == Balloon estimators == A common method of varying the kernel width is to make it inversely proportional to the density at the test point: h = k [ n P ( x → ) ] 1 / D {\displaystyle h={\frac {k}{\left[nP({\vec {x}})\right]^{1/D}}}} where k is a constant. If we back-substitute the estimated PDF, and assuming a Gaussian kernel function, we can show that W is a constant: W = k D ( 2 π ) D / 2 {\displaystyle W=k^{D}(2\pi )^{D/2}} A similar derivation holds for any kernel whose normalising function is of the order hD, although with a different constant factor in place of the (2 π)D/2 term. This produces a generalization of the k-nearest neighbour algorithm. That is, a uniform kernel function will return the KNN technique. There are two components to the error: a variance term and a bias term. The variance term is given as: e 1 = P ∫ K 2 n h D {\displaystyle e_{1}={\frac {P\int K^{2}}{nh^{D}}}} . The bias term is found by evaluating the approximated function in the limit as the kernel width becomes much larger than the sample spacing. By using a Taylor expansion for the real function, the bias term drops out: e 2 = h 2 n ∇ 2 P {\displaystyle e_{2}={\frac {h^{2}}{n}}\nabla ^{2}P} An optimal kernel width that minimizes the error of each estimate can thus be derived. == Use for statistical classification == The method is particularly effective when applied to statistical classification. There are two ways we can proceed: the first is to compute the PDFs of each class separately, using different bandwidth parameters, and then compare them as in Taylor. Alternatively, we can divide up the sum based on the class of each sample: P ( j , x → ) ≈ 1 n ∑ i = 1 , c i = j n w i {\displaystyle P(j,{\vec {x}})\approx {\frac {1}{n}}\sum _{i=1,c_{i}=j}^{n}w_{i}} where ci is the class of the ith sample. The class of the test point may be estimated through maximum likelihood.